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FOREWORD 


The American Society for Quality Control is 
pleased to publish these Transactions of papers 
presented at our 32nd Annual Technical Conference 
in Chicago, Illinois, May 8-10, 1978. Although 
these papers have been screened by the sponsor- 
ing Division or Technical Committee, the Society 
assumes no responsibility for the contents of the 
papers printed here. Some of these papers may 
later be reviewed by an editorial board of the 
Society and may appear in one of our regular 
publications. We hope that you will find these 
papers both educational and inspirational. 


Lawrence R. Dor sky, 

Vice President, Publications 



COMMENTS - 32nd ANNUAL TECHNICAL CONFERENCE 


Jay W. Leek 

Technical Program Chairman 


Quality--True Test of Management, the Theme for this year's Conference is indeed appro- 
priate and timely. The era in which we find ourselves challenges our management skills 
in dealing with the very basics of our social and industrial life. 

The seemingly endless degradation of family life, the continuous social upheaval and 
encounters, environmental exploitation, governmental control, and industrial problems 
pose unique but certainly rewarding challenges. Some of these we must face in the next 
decade include: 

a. Deciding the role we can play in our industrial society. 

b. Selecting operating goals which support the "Good Customer" concept. 

c. Understanding management actions implicit in the formula for business "Revenue 
minus Cost equals Profits" and the skills and techniques necessary to fulfill this 
promise. 

d. Planning for lower administrative costs and managing throughout to attain lower 
working capital requirements. 

e. Developing organizations utilizing people having contemporary educations, humani- 
tarian ethos, and goals not weighted heavily in money and materialistic incomes. 

f. Learning to be an effective agent of change. To go on as we are brings additional 
problems. 

Strategic planning today needs more than the normal lip service. Quality and Reli- 
ability professionals need to change their image from the tactician to strategists, 
more skilled in forward planning and thinking. Doing so will find a truly supportive 
management in attaining our goals and objectives and will assist in greater self- 
actualization, both in our social and industrial endeavors. 

The Committee joins me in hoping you will find our Conference informative, educational, 
stimulating and satisfying. The Committee has done an outstanding job in its prepar- 
ation and I would like to use this media to extend my heartfelt thanks for their out- 
standing support and excellent cooperation. 

Welcome to Chicago and the Palmer House and trust your stay will be enjoyable and this 
Conference the best ever. 
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QUALITY MODELING 


Raymond Goldstein, Staff Engineer 
Product Assurance, Engineering Directorate 
Litton Data Systems 
Van Nuys, California 91409 


ABSTRACT 

A model is suggested for quantifying a generalized manufacturing process unit as a function of time. 
Through a series of such interconnected models, a total manufacturing operation leading to the deliverable 
product may be depicted. A graphical interpretation of acceptables and defectives is presented. The key 
quality parameter is process gain - the ratio of acceptables to defectives. Analysis of model structures is 
carried out by Laplace transform methodology. 

TEXT 

System analysis literature is replete with the application of electrical equivalents as an interpretive vehi- 
cle for the modeling and performance prediction of mechanical, electromechanical and hydraulic systems. 
Assigning transfer functions representing the analog of each uniquely contributing member of a system facil- 
itates both analysis and physical understanding of the system dynamics. 

A quality model for a manufacturing process unit may be constmcted through such a series of elec- 
tronic circuit analogies. The kernel of the quality model - - the Process Unit - - is comprised of the least 
number of specific operations required to produce a definable product — a process followed by an inspection. 

As a preliminary step, it is essential that the quality model be placed in its proper perspective in the 
life cycle of a product. In the general case, it may be assumed that every process will produce an output 
comprised of acceptables and defectives. Since no process is without flaws, there is only a design-intended 
probability that it will produce a yield comprised principally of acceptables. In this context, the following 
definitions are suggested: 

Quality Control is a system of regulatory controls designed to maximize the probability that a 

product will conform to its specification requirements. 

Product Assurance, which encompasses Quality Control and Reliability Control, is now defined 

to provide an interlocking relation between both: 

Product Assurance maximizes the probability that a product manufactured in conform- 
ance with its specification requirements will continue to meet those requirements under 
prescribed operational conditions for a specified time. 

Quality is a specified group of quantifiable characteristics that determine the probability of a product 
conforming to its specification requirements. 

The Quality Control time span influencing the manufacturing process may be considered small by com- 
parison to the complete life cycle of a product, with the desired objective of MTBF » fabrication time 
(Figure 1). An exception exists for certain specialized products such as explosives, where fabrication time 
» MTBF. 

To make the definitions of Quality Control and Product Assurance statistically general, their respective 
probabilities may be further qualified by confidence levels. 

The significance of quality modeling rests in its quantification of the relation between process yield and 
attrition. Quality engineers are concerned with achieving satisfactory process yields, and are chartered to 
analyze and correct processes falling short of expectations. Materiel personnel share these concerns, and are 
not only mandated to supply the commodities listed on each bill of material, but are required to anticipate 
projected losses, and extend their purchases accordingly to maintain the material requirements for each pro- 
cess. Through a better understanding of process quality dynamics and their interrelation, the best intentions 
of Pareto may be served in terms of prioritizing analysis and correction action, and quality engineers may 
eventually be able to assist Materiel personnel to more accurately forecast their requirements. Similarly, 
since the correction of defective material entails additional manpower, assistance may ultimately be rendered 
managers concerned with the impact of defective material on personnel requirements. 
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Postulate 8, Inspection 

Perfect inspection is analogous to a phase-sensitive detector in that it separates P into its P a and P t j 
(in-phase and quadrature) components. 

Postulate 9. Generalized Operational Statement 

In that the probability exists that all operational steps leading to an accepted product may exhibit 
some degree of imperfection in producing defectives as well as acceptables, vector notation may be employed 
to characterize each operation, leading in sum total to the accepted product. 

Postulate 10. Generation of Defectives 


Defectives may be generated through the input of nonconforming materials to a process (external), or 
through process deficiencies (internal). 

Postulate 11. Analysis and Corrective Action (H term) 

Analysis and corrective action (ACA) provides negative feedback to a process so as to exclusively oper- 
ate on and attenuate its defective (P t j) output. 

Postulate 12. Rework and Repair 

Rework and repair operate on recoverable defectives, rotating the P<j r vector ~ radians to the P a axis. 
Postulate 13. Negative Acceptables (P an ) 

Negative acceptables result from the erroneous identification of good material as bad material (Type I 
error). 

Postulate 14. Negative Defectives (P(j n ) 

Negative defectives result from the erroneous identification of bad material as good material (Type II 
error). 

Postulate 15. Start-up Delay 

The start-up delay of a process is defined as the transient time between the start of a process and the 
time when the process output per unit time interval becomes constant. Included in this time are such fac- 
tors as operator learning curve, adjustment of production machinery, and corrections to drawings and/or 
specifications. 


Postulate 16. Processing Time 

Processing time is defined as the period required to 
produce one acceptable, following the start-up delay of 
a process (Figure 3). 

Figure 3. 



Postulate 17. Process Function 

Assuming a constant input of material, a process performs the operation o f a converter/integrator, pro- 
ducing a linearly rising output as a function of time, in assembling raw material to meet product specifica- 
tions. Utilizing postulates 2, 5, 7 13 and 14, the output of a process unit is graphically depicted (Figure 4). 



Upon correction of the Type I and Type Ii 
errors, and separation of defectives into 
recoverable and no_n -recoverable categories, 
P a is increased by P^j. and p an> and P,j is 
reduced by P^j., and increased by P ( j n 
(Figure 5): 



where P c and 6 C are the corrected process unit output and phase angle. The output of a series of such cor- 
rected process units may be combined as (Figure 6 ): 



m _____ _ _ m 

Where n P = Pi • P 2 • P 3 P m , and 11 P represents the end item derived from sequential 

n=l n=l 


process units. 


The processing function may be modeled 
as (Figure 7); 
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OUTPUT O 


Figure 7. 

Where G is the process gain and s is the Laplace transform operator. Assuming zero initial conditions, and 
a step input of material , O(s) = j , and O(t) = lGt . 

O(t) = IGt, is a function describing a linear integrator whose output is proportional to the material 
input, the process conversion gain, and increases with time. Postulate 15, Start-Up Delay, is omitted in the 
interests of simplicity, though in practice, the linear Processing Time (Postulate 16) is not achieved immedi- 
ately, unless the product had been produced for a period sufficient to account for all Start-Up Delay factors. 
A more rigorous treatment includes a representative start up function, O = IGt^, which prevails until its 
slope matches that of the integrating function s t 2 denotes an accelerated learning curve (Figure 8 ): 


For O = IGt 2 , ^ = 2 IGt. 






The slope of the integrator is ^ = 2IG. When 

1 dO ^ 

t = = IG, and the functions are joined 

at O = IG ( j ) =-~- (Figure 8): 



Figure 8. 


Inspection may be modeled as 
(Figure 9): 
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Figure 9. 


In terms of its electronic analogy, the process output consisting of quadrature and in-phase components 
(P a + 7 d ), is fed to a phase-sensitive detector. Process drawings and specifications are equivalent to the 
reference voltage in the phase-sensitive detector, and supply the baseline through which the separation is 
made of the process output into acceptables and defectives. 


Assuming perfect inspection, the acceptables are routed to the next product assembly operation, and 
the defectives are routed to the ACA operation (H), whether they may be due to external or internal causes. 
The output of the H operation is utilized as a source of negative feedback for defective material inputted to 
a process, and as an attenuator of internally generated defectives. Because of the unique property of per- 
fect inspection to separate a process cJutput into its P a and P d components, negative feedback is selectively 
applied, so as to reduce P d , while leaving P a undisturbed. The simplified model depicting the II function is 
(Figure 10): 



When H » 1, ^ may be simplified as: ~= 

I I H 


indicating that the process output of defectives is 


inversely proportional to the effectiveness of corrective action. With the defectives attenuated by a factor 


of ^ , the new process gain, improved by ACA, is: G'= H 
ACA feedback is now developed for the acceptable and defective output components. 


The simplified process unit model with 


O '-'a 

Considering the acceptable part of the output: — = * — where G„ is that part of the process producing 


'-'a j 

P a acceptables, and O = —I. For a step input of — , taking the inverse transform: 


1 G a 

O(s) , and O(t) — IG a t, 


where the acceptable process output increases linearly with time, and is proportional to the material input 


and process conversion gain. Considering the defective part of the output: 


O 


1 s where G d is that part 
of the process producing P d defectives. In this case, the discriminator characteristic of inspection will pro- 
vide a feedback signal to the ACA (H) function, and the loop will be closed for defectives. For a step 

P d G d 


input of — , O(s) =■ 


P d G d 


1 + 


G d H s(s + G d H) ' 


Taking the inverse transform: 


O(t) = P d G d 


1 - e 


- G d Ht 


1 - e 


g,,ht 


O(t) = 





As t increases definitely, the exponential term may be disregarded, and O(t)-*- — - . The same result 

H 

could have obtained using the final value theorem for Laplace transforms: the process output of defectives 

P 

is an exponential function, asymptotic to — (Figure 11): 

H 



Figure 1 1 . 


Note that the H term is a function of the effectiveness of ACA, and is measured by its ability to 
attenuate defectives. 


As before, the adjusted process gain with ACA becomes: G' = H 
and repair is (Figure 12): 

PROCESS INSPECTION! 


The model depicting rework 



Figure 12. 

The P t j defectives have been converted by the Rework and Repair integrating function_into acceptables 
and non-recoverable defectives (scrap). The P(j r component is summed _up with file initial P u component 
and the Pan negative acceptables to form the total acceptable output DP = P a + P^r + Pan- Providing it is 
cost effective, P t j s may be subjected to one or more iterations through the Rework and Repair cycle in 
effort to reduce the amount of scrap and obtain a higher yield of acceptables. 

Refining the model further, it is observed that the attenuation of defectives does not occur immediate- 
ly, as a finite period (T) is required to analyze the cause of defectives and develop responsive corrective 
action. The transfer function for ACA is of the form He"^ s where T is the dead time prior to the attenua- 
tion of defectives. 

The inspection function does not occur immediately either, but is subject to a lag time constant r, and 
a transfer function of the form j • As inspection rarely generates defectives, it is considered a unity 
gain function. 

Connecting the transfer function building blocks to form the complete model (Figure 13): 



CORRECTIVE ACTION 


ACCEPTABLE 
OUTPUT 0 


Figure 13. 

As the dead time required to complete the analysis of defectives and develop the required corrective 
action is equivalent to opening the ACA feedback loop, the defectives will not be attenuated during the 
period T, and consequently, the process gain will not be improved during the interval. The output of the 
process unit can be developed as the sum of two time dependent functions: that without ACA, followed 
by that with ACA. 
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During the interval T, when ACA is under development: 


O, , G 1 

T <s) ITT; 
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r 

O(t) may be approximated as: O(t) « IGt, an integrating function where O(t) 38 IGT at the end of the 
interval T. 

The output as a function of time is simply: |P| = |P a j + |P(j| where the process unit output is deter- 

I p a I 

mined by the material input I, and the process unit gain G = — . 


Upon completion of the ACA interval, the H feedback loop is closed and the response for — (s) is in 
41 r 0 /N G(s) 1 
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Substituting transfer functions: y(s) = — j — “ + TS ) + GH 
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Rearranging terms and multi- 


0 1 r 

plying and dividing by H: (s) = — •— — - — Which is in the form: — 
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Depending upon the values of G, H and r, the quadratic function may 


assume various degrees of damping. If the inspection time constant r is large, the process unit will exhibit 
an underdamped condition. The economics of most manufacturing operations generally dictate that r be 
small compared to the unit fabrication time. 


The inverse transform for the quadratic is of the form: 


T (,) = H 


Vi -? 2 


O 

With the assumption that the process is at least critically damped (f = 1) , — (s) and 

2 ‘ H(S + “"> 2 

0 co n \ e n 

T (t) . The rework and repair operation may be treated in the same manner as a process 

1 H 

unit without ACA feedback, assuming a small inspection time constant r . 


Similarly, — (t) ^ IG r t, where I is the Pj component consisting of recoverable and non-recoverable 
I 1 

defectives. It is noted that the rework and repair input of defectives will be reduced by a factor - follow- 

ri 

ing the time when the ACA loop is closed, by comparison to that during the interval T. Summing up the 
process unit, and the rework and repair functions: at the completion of time interval T, the process loop 

transfer function for recoverable defectives is: ^(s) = —\r~ — V — land ^(s) =—^(—-7 — ] is the 

I Ml + rsl s ll + rsl I s 11 + rsl 

transfer function for the acceptable output. ' ' ' 


and ^(s) =- 



Summing up both transforms: 


0 
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Through the preceding methodology, additional functions may be derived that depict other segments of a 
manufacturing operation. 


The stability of a system may be treated by superimposing its output on a control chart (Figure 14): 


UPPER CONTROL LIMIT 

LOWER CONTROL LIMIT 



CENTERLINE 


Figure 14. 


The transient stability margin of a system may be calculated as the distance between the inflection 
point, and its nearest control limit: 


Transient stability = 


d 

(UCL - LCL) 
o 


The long term stability margin may be calculated as follows (Figure 1 5): 


UPPER CONTROL LIMIT UCL 


LOWER CONTROL LIMIT LCL 
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Figure 15. 

UCL - y. 

For this situation, At = t 2 - tj = — — . With a slope m, the process output will intersect the UCL at 

t 2 » if left uncorrected. At is the “time to go” before violating the UCL, indicating the time available for 
ACA to take effect, if the UCL is to be avoided. 

While closed loop analysis may utilize many different methods to determine system stability, the criter- 
ia for a stable system as applied to a Process Unit has a different interpretation, i.e.,the output may be con- 
tinuously oscillating, but providing it is confined to the region between the LCL and UCL, the process must 
be considered stable in terms of its ability to produce an acceptable product. A continuously oscillating 
servo system would normally be considered indicative of an unstable, and hence, unacceptable condition. 

In the sense that the acceptability of a Process Unit is measured by its ability to remain between the 
UCL and LCL, as prescribed by specifications, generically, it resembles a regulating system more than con- 
tinuously variable servo system. 

ADDITIONAL OBSERVATIONS 


1. Further consideration of the inspection function indicates that is should include a defect leakage 
path to account for less than perfect inspection (Figure 16): 
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where &■ is a function representing the residual defectives after attenuation by inspection, excaping detection 
and separation into the and components (Postulate 6). 


2. In considering the difference between externally and internally caused defectives, a more rigorous 
model of a process is depicted (Figure 17): 


PROCESS 



The process input is the sum of acceptable and defective material. Similarly, the output is the sum of 
acceptables and defectives. The P^ output component includes both externally caused defectives due to the 
unplanned input of nonconforming material, plus internally generated defectives due to process imperfections. 

3. Though the separation of the Process Unit output into acceptables and defectives has been hereto- 
fore assigned exclusively to the inspection function, in practice the operator assigned to perform or monitor 
the process operation contributes to the inspection function through observing anomalies in the process out- 
put. Thus, some degree of defect reduction may actually occur prior to the formal inspection function. 

E.O. Codier (Ref. 1) states that ^p<jrt total ^intrinsic ^induced* ^tid believes that ^induced ^ rnuch 
larger than /''■intrinsic • Possible contributions to ^induced are: 

Lack of regularly scheduled operator training and certification 

Unauthorized departures from manufacturing instructions 

Manufacturing instructions that do not reflect component application constraints 

Insufficient process controls 

Arbitrary selection of sampling plans without regard to the probability of accepting a specific 

percentage of defectives 

Failure to recognize that repaired or reworked assemblies may have degraded reliability 

Inspection oversights 

Inadequate specification controls of critical parameters 

Lack of verification of stress levels. 

In particular, the contribution of each of the above to the part total failure rate are unique to each 
company, and comprise part of its “quality fingerprint.” As a functional tool, it is suggested that the fail- 
ure rates for each predictive entity be increased by a factor related to the percent rejections of each compo- 
nent by comparison to the using population. Should any test failure occur resulting in a component replace- 
ment or assembly repair, a failure rate penalty would be exacted. Thus, only those components that remain 
undisturbed in the entire manufacturing cycle would be characterized by the intrinsic failure rate. 

4. Failure Modes and Effects Analysis (FMEA) is a recognized method for revealing sources of design 
weakness affecting reliability. The FMEA consists of a review of component, assembly, and system design 
characteristics in terms of their vulnerability, and indicates the most likely effect in the light of possible 
failures. As the study is performed by reliability and design engineering, recognition of sources of degrada- 
tion incurred during the Product Fabrication Regime is almost nil. There is a need to identify process, 
assembly, inspection and test problems that may compromise system reliability during field deployment, in. 
addition to those areas that are influenced by engineering design. 

5. Bum-in, as depicted in Figure 1 indicates that useful product life starts with its completion, follow- 
ed by customer acceptance. Its purpose is to surface sources of infant mortality prior to customer delivery. 
Premature failures after customer receipt raises questions as to the sufficiency and effectiveness of burn-in. 
The presence of latent defects that become obvious only after time and use indicates the need for more 
effective screens during the Product Fabrication Regime. Field failure data should be fed back to augment 
the FMEA, and provide guidance as to where additional screening can be most effective. 

A brief insight has been presented into the Quality aspects of the product life cycle. The omission of 
these factors may account for part of the difference between predicted and operational reliability. The 
“parts count,” failure rate prediction, except for such relatively few considerations as the failure rate for 
connections, PC boards and a “quality factor,” ir applied to failure rates, infers through omission of the 
manufacturing function that it has negligible influence on achieved reliability. The impact of rework and 
repair, particularly when catastrophically failed components may cause secondary overstress, is generally 
neglected for MTBF predictions. It is not reasonable to expect an assembly that has been repaired a num- 
ber of times to exhibit the same reliability as an undisturbed assembly. 




The tacit reliance on downstream assembly, system test and inspection to screen defectives places an 
excessive demand on these functions, increasing the probability of higher order defect escapes due to imper- 
fections in these operations. The initial quality conditions caused by manufacturing variability must be 
included in reliability predictions, to complement the parts count — failure rate methodology. 

With the ever increasing emphasis on product liability, and the military now focusing on Reliability 
Improvement Warranty (RIW), the successful evolution of a product must, of necessity, be considered from 
the viewpoint of Product Assurance, rather than the heretofore separate entities of Quality and Reliability. 
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VENDOR QUALITY ASSURANCE AND RELIABILITY 


Paul J. Breibach 

Director Corporate Quality Control Incoming Materials 
Zenith Radio Corporation 
6001 W. Dickens, Chicago, Illinois 60639 


In today's highly competitive home entertainment electronics market, the name of 
the game as far as the end customer is concerned is a more reliable product at a 
lesser cost. 

It becomes an ever increasing search on the part of the producer to meet the 
challenges of higher quality and reliability demanded by today's customers while still 
holding the cost down. 

The first question, of course, is what do these demands do to the vendor cost and 
what does this do to the profit picture of the Corporation. For as we all know, any 
company, to stay in business, must realize a reasonable profit. 

One of the most fertile areas in which to work to meet the demands of higher 

quality and reliability is in the product or component as received from the vendor. 

Over the years, it has been common practice that when a manufacturer purchased a 
component, he would inspect it to insure that it conformed to the print and/or speci- 
fication before placing the item into his production facilities. 

If the component has been properly designed by the manufacturer’s Engineering 
Department, and the parameters and tolerances are realistic for the end quality and 

reliability desired, he should not have to inspect the item as received if both the 

vendor and the manufacturer have firm understanding of what is desired. 

Zenith has a program which is known as the Zenith "Vendor Quality Assurance 
Reliability Program". It is the purpose of this Program to work closely with Design 
Engineering, Purchasing, Manufacturing, and the vendor in welding a team which would 
have the same end objectives and offer benefits to both Zenith and its vendors. 

It was decided that the classification of defects and the quality levels for our 
Program would be 0% for critical which is safety related defects, and .65% for majors 
and 1% for minors. 

The first step in the Vendor Quality Assurance Program is that of qualification 
to determine if the vendor is qualified to produce the component per the print and/or 
specification set forth by Design Engineering. For it is our belief that unless the 
vendor is qualified; that is, he has the proper facilities, test equipment, technical 
capabilities, expertise, he will not consistently produce the component to the quality 
and reliability demanded by Zenith. 

The Zenith Vendor Quality Assurance Program works as follows: To introduce a 
vendor to the Program, the Zenith Purchasing Manager extends an invitation to the 
representatives of the vendor's Management, Quality Control, Marketing and Manufactur- 
ing to visit Zenith and meet with the Purchasing Manager, a Vendor Quality Assurance 
Engineer, and a Design Engineer. At this meeting, the Vendor Program is explained by 
a visual, oral slide presentation, pointing out the benefits that can be received by 
both the vendor and Zenith. As our program progressed and to reach more vendors in a 
shorter period of time, we held group vendor meetings consisting of 15 to 20 vendors 
at a time. 

The first thing explained at this meeting is the Zenith AQL levels and classifi- 
cation of defects. We require our vendors to certify to 0% for critical failures 
(safety related parameters) and .65% for major defects and 1% for minors. Following 
this explanation, the vendors are given a series of questionnaires and requested to 
fill these out so that we may have a record of his facilities and equipment. These 
questionnaires are designed so as to have the vendor show the type of equipment he 
has, its accuracy and whether the equipment is used in Production, Engineering, or in 
Environmental type setups. Also it is explained to him at this meeting that once he 
is qualified, a Correlation Program will take place wherein he will measure a given 
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number of components he manufactures for Zenith, recording the exact data for the 
parameters agreed upon and then the data and measured components are submitted to the 
Zenith Incoming Quality Control Department at the using plant for their evaluation 
and correlation of the data. 

Once correlation is established, the vendor is requested to periodically submit 
correlation samples with appropriate data in with incoming shipments so that a contin- 
uous correlation can be maintained. 

Following correlation, the vendor is requested to supply certification sheets 
with each incoming shipment. This is not merely a letter stating that the vendor 
certifies the shipment per the Zenith print and/or specification, it should be a copy 
of his outgoing quality inspect ion showing the number of samples taken and the number 
of defects found for each parameter measured and the AQL levels to which he inspects. 

Following this meeting, the vendor is informed that when we receive the completed 
questionnaires, and they are reviewed and found to be complete, an appointment will be 
made for a Zenith Survey Team; consisting of: Purchasing, Design Engineering, when 
required, and Vendor Quality Assurance to visit his facilities. It is the objective 
of this Survey Team to meet with the vendor's top personnel from such departments as: 
Management, Marketing, Production, Purchasing, Quality Control and Engineering to 
again explain the entire Zenith Vendor Quality Assurance Program and the Zenith AQL 
levels and classification of defects. 

After this meeting, the vendor's personnel and the Survey Team tour the vendor's 
facilities in order to assure themselves that the vendor is capable of manufacturing a 
component with good reliability and high quality to the Zenith print and/or specifica- 
tion. Following the tour, again a meeting is held with the vendor's top personnel so 
that any questions on either the part of the Survey Team or the vendor, may be 
answered. The Survey Team requests from the vendor once each month a copy of his 
monthly Environmental Test data and/or life test results. In a business such as 
Zenith is engaged in, if our Environmental Test Lab were ten times its present size, 
we could not possibly environmentally check every component every month. However, 
since the vendor is doing this as a continuous control within his facilities, we have 
found him to be most happy to supply us with carbon copies of his results. This gives 
us an additional control in our Program. 

If the Survey Team is satisfied that the vendor is capable of meeting the require- 
ments of the Zenith Vendor Quality Assurance Program, they request a letter from an 
officer of the company, stating that he, the vendor, is in agreement with the Program 
and will work with Zenith in making the Program a success. 

The correlation part of the Program is put into effect following this meeting and 
as soon as correlation is completed, the vendor starts to certify his incoming ship- 
ments. Attached is a copy of the questionnaires used, letters mailed to the vendors 
along with samples of correlation sheets and suggested certification sheets. This 
Program has been so designed as to be flexible in the area of obtaining needed data. 

We have found that if we would demand of the vendor that he record data on specific 
data sheets, we would most assuredly increase the cost of the component received by 
Zenith. Therefore, we are always open to using the vendor's data sheets if it is 
complete and gives the information desired. However, in many cases if the vendor does 
not have adequate data sheets, he is most anxious to accept the one suggested by 
Zenith. 

One area which we have found to be extremely important in this Program is that of 
lot numbering or identifying each incoming shipment. The vendor either starts a lot 
numbering system or in many cases, he has been able to use his Shop Order number as a 
lot number and this gives an excellent control since we can trace it back to when the 
job started. The only restriction to the lot number is that it must not repeat itself 
in less than 18 months. 

The real importance of the lot number to Zenith is that when a quality problem is 
found in Production, we are able to trace it and not have to go through all the 
material in our house. Again, it gives us an excellent control in this Program and a 
means of tracing information between the vendor and Zenith. 





Now that the vendor has been qualified, and correlation is established, he is 
ready to enter into the certification of incoming shipments. As was explained 
earlier, his incoming shipments must be accompanied with a copy of his outgoing 
quality control sheet showing the amount of samples taken and rejects found. This 
sheet should be signed by the Quality Control Manager of the company. This then 
becomes the certification document. The first five shipments received at Zenith are 
inspected in our normal incoming inspection procedure which uses MIL STD. 105D Level 
II. If we find the first five shipments to meet o;ir AQL's and the data sheets to be 
complete, we then certify the vendor for that particular part number only. Here, I 
would like to point out that we certify our vendors by PART NUMBER . We qualify a 
source but we correlate and certify by part number only. Following these first five 
shipments. Incoming Inspection then has the prerogative of inspecting any one out of 
the next five shipments providing each shipment has an adequate data sheet accompany- 
ing it. However, Incoming Inspection at Zenith is NOT permitted to skip lot any 
"safety related parameters". All safety related parameters must be inspected without 
exception. The shipment to be inspected is selected at random, usually by throwing a 

die and in this manner, the vendors have no idea as to which shipment will be 

inspected. However, Zenith, of course, has the prerogative of inspecting each ship- 
ment as it is received on either a full or a sampling basis. 

A vendor continues in the manner described above until a quality problem is found 
either by Incoming Inspection in the inspecting of one of the lots or if a latent 
quality or reliability problem is found in the production lines on a shipment which 
was passed under certification. When this condition arises, the vendor is removed 
from certification, notified of the quality problem, and all incoming shipments are 
inspected using MIL STD. 105D Level II. 

Now, for him to be recertified, he must first submit documentary proof that he 
has corrected the problem and again, we must receive five consecutive shipments which 
will be inspected under the normal sampling plan of MIL STD. 105D Level II without 

rejection along with complete certification sheets. If all goes well with these five 

shipments, he will again become certified for that part number. 

We here at; Zenith have a group of graduate Engineers known as Vendor Quality 
Assurance. It is the responsibility of this group of Engineers to work with the 
vendors in establishing of the Zenith Vendor Quality Assurance Program in their plants 
and to work closely with vendors when quality problems arise. So, when a vendor is 
removed from certification, one of these Engineers works with him in the elimination 
of the problem and assists him in becoming recertified. 

This is the explanation of the way Zenith works in the control of their vendor’s 
quality. However, we feel that any Program such as this must have certain benefits 
for both parties concerned for it to be a success. Some of the benefits which we 
have noted are: 

A. Improved quality. 

B. Improved reliability. 

C. Faster approvals. 

D. Fewer rejections. 

E. Saving of environmental test time. 

F. . Better assurance of the vendor's quality. 

G. Vendor better informed of what is expected of him and his product. 

H. Through the reporting of control dimensions, he will know where to concen- 
trate his inspection to a minimum while still assuring component of the 
desired quality and reliability required by the Engineering specification. 

I. Shipping cost savings can be realized by the assurance of quality and 
acceptance of his material. 

J. Rework and scrap costs will be minimized. 

K. By the use of recognized quality control techniques, he will have reliable 
records of his quality. 


tiwai©! t&iviasav 



L. Better vendor-Zenith relations. 

K. Zenith benefits by: The assurance of better incoming quality and reliability, 
better quality of our manufactured product and increased sales which is also 
a vendor benefit. 

In addition to these, it is our feeling that every program such as this should 
have some incentive to arouse the interest of the vendor's people in the Program. As 
a result, Zenith initiated a Vendor Quality Certification Award. This award is a 
means of informing the vendor and his personnel that they are doing a good conscien- 
tious quality job for Zenith and the Vendor Quality Assurance Program. The award is 
presented twice per year in January and July. The award is signed by our Vice 
President of Manufacturing, Vice President of Purchasing Department, Executive Director 
of Purchasing, Director of Corporate Quality Control, Director of Incoming Quality 
Control, and Manager of Vendor Quality Assurance. 

The program works as follows: 

VENDOR PERFORMANCE RECOGNITION AWARD PROGRAM 


Purpose 

The Vendor Recognition Award Program is designed to recognize those vendors who 
deliver material of above average quality to Zenith Radio Corporation over an 
extended period of time. The two levels of awards are intended to motivate above 
average (B) vendors to attain an excellent (A) rating. 

Eligibil ity 

1. The vendor must have been an active Zenith vendor for more than one year. 

2. The vendor must have delivered at least 6 lots of productive material to 
Zenith during the six month rating period. 

3. The vendor must be in compliance with the Zenith Radio Corporation Vendor 
Quality Assurance/Reliability Program. 

4. The vendor must not have had any major in-process quality or reliability 
problems which were due to vendor responsible failures in the product during 
the award period. 

Awards 

1. A "Gold” Recognition Award will be presented to those vendors who have a 
rating of A for all parts supplied as reported by all Zenith IQC Departments 
during a given award period via the Vendor Analysis System. The requirements 
for an A rating are as shown below. 

2. A "Green" Recognition Award will be presented to those vendors who have an 
overall rating of A but have some specific parts with B ratings as reported 
by all Zenith IQC Departments during a given award period via the Vendor 
Analysis System. 

Note: An A vendor rating is obtained by having a performance figure of 1% or 

less based on the following formula. 

Lot Rejection Percentage + 4 (Process Average) 

Vendor Rating = 5 

AA defects + A defects 

Process Average (P) = Total Samples 

Procedure 

1. At the end of June and December of each year, the Vendor Quality Assurance 
Department will get an overall performance rating for all Zenith vendors. 

2. These overall ratings will be reviewed per the above eligibility rules and 
a list of eligible vendors will be formulated. 

3. The list of eligible vendors will be reviewed by the Vendor Quality 
Assurance Engineers to be sure that Zenith is not having in-process or 
reliability problems with any of the eligible vendors' products. 



4. The award eligibility list will then be submitted to the Purchasing Managers 
for their review and approval. 

5. The list of eligible vendors will then be returned to Vendor Quality 
Assurance and the appropriate award will be generated. 

6. Awards will be presented at Zenith and/or vendor's facility as appropriate 
within 30 work days after the close of the rating period (six months). 

We also have a "Zenith Computerized Vendor Rating Program" which is so designed 
as to supply to our vendors a monthly computer report which shows his quality 
performance as seen by Incoming Quality Control (incoming inspection) Departments. 

The Program operates as follows. 

ZENITH COMPUTERIZED VENDOR RATING PROGRAM 

Once each month our vendors will receive in the mail a copy of the "Zenith Radio 
Corporation Vendor Performance Report". This report will show by part number the 
quality status of the materials shipped by the vendor to our Zenith facility for the 
previous month. 

Attached is an example of the report the vendor will be receiving and reads as 
follows . 

Item A - Company name and Zenith vendor number. 

Item B - The part number(s) vendor has shipped to a Zenith facility in the 
previous month. 

Item C - Week ending, designates time period in which material was received and 
inspected. 

Item D - Shows number of lots inspected, number of lots rejected, percent rejected 
and the number of rejected lots returned to the vendor. If there is a 
difference between the number of lots rejected and the number of lots 
returned to the vendor, this would mean Zenith elected to use these, lots 
by 100% sorting, reworking, or by deviation. 

In any of these cases, the vendor would have been notified by the Vendor 
Quality Control Engineer as to our decision and the action taken to use 
the material in production. 

Item E - Shows number of parts received and number of parts rejected. 

Item F - Shows number of random samples taken per MIL-STD-105D Level II single 
sampling, from the lots inspected (reference Item D) . 

Item G - Shows total defects found in the samples inspected. The three classes 
of defects shown are: 

"AA" - Critical or safety related. 

"B" - Major defect. 

"C" - Minor defect. 

Item H - Shows the process average for the total samples inspected. The process 
average is established by means of the following formula. 

"AA" + "A" 


Process Average = Total Samples 



Item I - Shows approval status for components vendors ship to Zenith: 

A - Approved. 

L - Limited Approval. 

N - Not Approved. 

W - Approval Withdrawn. 

Item J - Shows Vendor Rating Code. 

A - Excellent. 

B - Good. 

C - Average. 

D - Poor. 

E - Unsatisfactory. 

This then is the basics on which Zenith Vendor Quality Assurance Program operates. 
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ZENITH RADIO CORPORATION 

CORPORATE QUALITY CONTROL 

VENDOR QUALITY/RELIABILITY EVALUATION REPORT 

DATE 

PREPARED BY 

COMPANY NAME 

Dl Vl St ON O F 

| STREET ADDRESS ClTY STATE 

PHONE NO. 

TYPE OF PRODUCT MADE 


QUALITY ASSURANCE 


DO YOU HAVE A QUALITY CONTROL DEPARTMENT? P YES l 2 N0 

DO YOU HAVE A QUALITY ASSURANCE DEPARTMENT? j I YES NO 

IF SO, TO WHOM DO THEY REPORT’ 


HOW DO THEY RELATE’ 


] SEPARATE OR P C OMBINED 

DO YOU HAVE A WRITTEN QC / QA RELIABILITY PROGRAM? YES Q NO 

_) ATTACH TO THIS REPORT, IF AVAILABLE, AN OVERALL COMPANY QUALITY ORGANIZATION CHART. 

I j ATTACH TO THIS REPORT, IF AVAILABLE, A Q.C. AND/OR Q.A. MANUAL. 

HAS THIS MANUAL BEEN COORDINATED WITH ANY GOVERNMENT AGENCIES? pf YES P N0 


INSPECTION CAPABILITIES 


DO YOU HAVE A FORMAL INCOMING INSPECTION FUNCTION? 

L2 YES 

n n ° 


.. . J 

I WHAT TYPES OP RECORDS ARE KEP T OF BOTH ACCEPTED AND 

REJEC TED MATER 

ALS? 


DO THEY HAVE AVAILABLE ALL SPECIFICATIONS AND DRAWINGS PERTAINING TO THE RECEIVED MATERIAL? 

r i yes m no 

IS THERE AN IN-PROCESS INSPECTION PROGRAM? 

□ YES 

rzi n ° 


IS THERE A FINAL INSPECTION PROGRAM? 

| “] YES 

NO 


IS THIS INSPECTION ROUTINELY MONITORED BY Q.A.? 

, ; YES 

P NO 


_____ j 

IlS THERE A FINAL INSPECTION OF PRODUCT? 

]D0ES FINAL Q.A. USE A SAMPLING PLAN’ 

\2 YES 
! ! YES 

H NO 
' j NO 


WHAT A Q L ' S ARC EMPLOY CD? 

ARE THERE ADEQUATE RECORDS OF INSPECTION RESULTS? 

P YES 

P NO 


whAT PRECAUTIONS APE USED TO ASSURE PROPER EN VI RONME 

VITAL CON TROL O 

RECEIVED MATERIALS? 


ARC THEY MAINTAINED WHEN MATERIAL »S RELEASED TO PROD 

LJCTION’ 
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A DEFECT REPORTING SYSTEM OF HONEYWELL 

Joseph P, Hodge, Quality Supervisor 
Honeywell Avionics Division 
St. Louis Park, Minnesota 55416 


. An effective quality organization ascertains defects or failures and their trends 
m a timely manner. Honeywell accomplishes this by a responsive reporting sys- 
and* w da TVn- b c: a Sn US ^ by *1! dis ^ linGS throughout all phases of production build 
electronic hardware 1 * deScrLbes Honeywell’s system used during build and test of 


HONEYWELL DEFECT REPORTING SYSTEM 


^ 11 +? efeC ^ R ?P° rt (HHR), Figure 1, is the basic system document. It 
is initiated by both production and inspection personnel when any defect or failure is 
observe during functional testing at all levels of assembly, including final device 
build level fhe HDR form is also used for return goods evaluation and subsequent 
repair Wien an anomaly is observed, the technician performing the test enters the 
description of the defect, the test during which the defect was noted, the environment 
(high, low, or room temperature), and the data on the HDR. When the cause of the 
lS JnS nd, TT the te . cbnician performing the analysis and troubleshooting com- 
pletes the HEIR. He includes such information as cause of defect, rework action 
required, and any recommended preventive action. He also lists the circuit location 
and part numbers of all components replaced and the reasons for replacement 


.TJ 1 ®, Gora -Pl-eted HDR is now separated; it is a three part form. The first copy along 
with defective components (attached) are sent to the responsible program Quality 
Engineer, the second copy is sent to the Production Foreman, and the third copy 
ln tbe device folder as a permanent record. Figure 2 shows the flow of 
the HDR system and the responsibilities of each discipline in the total system. 


The Production Foreman reviews his copy of the HDR for adverse trends 
operator performance problems, and completeness. His review could result in a 
number of actions -- 


Retraining /recertification of an operator 
Re-assignment of an operator 
Disciplinary action 

Request for engineering analysis of a process problem 
Request for engineering change to existing procedures 


Prodaction Foreman's review is important because he takes timely action 
workmanship problems or requests engineering assistance on more complex 
problems. 1 


The Quality Engineer, upon receipt of his copy of the HDR, discerns obvious 
adverse trends, process problems, and operator performance problems. To aid 
him in performing this function, a computer program has been established that 
processes information coded by the Quality Engineer from the HDR forms. An 
example computer printout is given in Figure 3. This program is run weekly and 
is distributed to cognizant Production, Reliability, and Design Engineers for review 
Also, upon request, a run can be made to cover any time frame desired (month 
year, program inception to date) for trending purposes. 

In reviewing the run, information such as program, device serial number, maior 
assembly, minor assembly, failed part, date, test level (where anomaly occurred) 
cause, circuit location, and HDR number are listed. Also the run can be sorted by 
any of the above elements. The example (Figure 3) is sorted by program, failed 
« a • CLrcult location. This computer program has been extremely valuable in 
identifying recurring failures due to lot-related piece part defects, process prob- 
lems, circuit application problems, and recurring workmanship defects. 
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The Quality Engineer works primarily with Reliability, Production and Design 
Engineers, the Production Foreman, Inspection Foreman, and Procurement Quality 
Assurance in resolving problem areas and obtaining expeditious and effective cor- 
rective action. Some typical actions that can result are: 

- Print changes 

- Process changes 

- Work instruction changes 

- Additional operator training 

- Vendor corrective action 

- Field retrofit 

- Continued monitoring for additional trend 

FAILURE REPORTING ANALYSIS AND CORRECTIVE 
ACTION SYSTEM (FRACA) 

The FRACA system, which is used in conjunction with the HDR system, is a 
closed- loop, documented corrective action system. A Failure Analysis Committee 
(FAC) comprised of members from Quality, Reliability, Design, and Production 
Engineering implement the system. When a potential trend problem is detected, 
formal failure analysis is performed isolating the cause of the failure and defining 
the necessary corrective action. This analysis and corrective action is documented, 
with concurrence required from all engineer disciplines on the FAC. Quality 
Engineering ensures implementation of the corrective action and monitors the 
effectiveness of this action through the IIDR system. In a review by a special 
NAV AIR- Reliability audit team in 1976, Honeywell was commended on having one of 
the best reporting and corrective action systems in industry. 

SALVAGE MONITORING 

Coupled with the HDR system, a computer program has also been established by 
Quality that provides, on a weekly basis, information on production salvage time 
expended during replacement of defective piece part components during all phases of 
the manufacturing process. Analysis has shown that the greatest amount of piece 
part salvage expended is limited to particular families of components, and Paretos' 
Principle of Maldistribution can be used to organize salvage information. This 
information aids in identifying high cost areas whore corrective action is required 
and also aids in evaluating the effectiveness of previous corrective actions. Figure 
4a is an example summary sheet, and Figure 4b is an example trend sheet for inte- 
grated circuits used on a specific program. 

SUMMARY 

Honeywell's Defect Reporting System is a key element of our quality program.. It 
is important not only in controlling salvage and scrap cost but has been vital in 
meeting fieLd performance criteria, thus avoiding costly retrofit and/or loss of 
future business because of poor performance. Through its conscientious use, it 
summarizes and identifies problem areas that can be monitored easily and contin- 
ually. • 
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RANDOM VIBRATION TESTS TO MIL-STD-781C 


Wayne Tustin, President 
Tustin Institute of Technology 
Santa Barbara, California 


INTRODUCTION 


This paper primarily affects quality and reliability organizations within the Navy 
and Air Force and within firms building (1) avionics equipments for use in jet aircraft 
and (2) airborne missiles, weapons and assembled external stores carried by jet air- 
craft. These organizations are strongly impacted by the random vibration requirements 

of the October 1 977~released Military Standard 781C^^ . Quality and reliability person- 
nel in other services and other firms should also be aware of random vibration. Their 
equipments, in transit and use, may experience little or no random vibration, but ran- 
dom vibration effectively screens against certain types of workmanship flaws. 

This paper does not deal with the reliability nor the statistical aspects of the 
Standard. It largely ignores the varying temperature and the humidities required for 
tests under -781C, also the sinusoidal tests required for fixed ground equipments and 
for equipments destined for transport and use aboard ground vehicles, ships, helicopters 
and turboprop aircraft. 

Rather, this paper emphasizes the random vibration tests which -781C requires for 
reliability demonstrat ion tests of avionics elements of jet-powered fighter aircraft, 
transports and bombers, also tests of air-launched weapons and external equipments. 

Certain figures and tables of -781C and other standards are reproduced here; they 
carry the numbering of those documents. Figures in this paper carry letters from A 
through C and the paper provides cross references to those documents. 


WHAT IS RANDOM VIBRATION? 

Let us first consider sinusoidal vibration and Fig- 
ure A, in which the sinusoid represents a time history, a 
record of instantaneous acceleration varying sinusoidally 
with time. The lower portion of Figure A is the corres- 
ponding spectrum; it shows that all the vibratory energy 
is concentrated at a single frequency f. For some tests, 
f is fixed. For other tests, f varies according to some 
program. Such tests generally only excite one specimen 
resonance at a time. 



Sinusoidal vibration is seldom found in the real 
world, and sinusoidal vibration tests are likely to be in- energy 
appropriate representat ions of the transportation and use 
environments. However, they may be appropriate for equip- 
ments going into fixed ground installations, on board 
ships and helicopters. 




FIGURE A 

By contrast, Sinusoidal Vibration 

consider Figure B, in which the time history rep- 
resents acceleration varying randomly with time. 
The corresponding spectrum shows that vibratory 
energy is spread out over a continuum, a contin- 
uous range of test frequencies. The principle 
advantage of random vibration testing is that all 
specimen resonance are excited simultaneously, 
much as occurs aboard h igh-performance aircraft 
and missiles in flight. 


FIGURE B 

Random Vibration 
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BACKGROUND TO THIS PAPER 



FIGURt L 

Vibration of forward-looking 
radar on A-7D/E air ora ft , 
RMS g vs. frequency in Hz 


What was wrong with the vibration tests required 
under MIL-STD-781 , -A and ~B? As suggested by Figure 
( 2 ) 

C, taken from Swett v , vibration (combined with vary- 
ing temperature) at an intensity of 2.2 g was applied 
at any optional fixed nonresonant (nondamaging) fre- 
quency between 20 and 60 Hz. Not surprisingly, test 
vi brat ion- i nduced failures have been rare, even with 
long-duration tests. But in-flight vibration aboard 
high-performance aircraft and free-flying missiles, 
also aboard captive-carried external weapons and 
stores, is known to relatively quickly induce fail- 
ures. Why the discrepancy? Figure 6 shows that 
in-flight vibration and gunf i re- i nduced vibration are 
broadband .The environmental broad-spectrum random vi- 
bration qualification tests exemplified by 

MIL- STD- 810^ are more realistic for this applica- 
tion; such spectra are more useful for reliability 
demonstration tests than was the original -781. 


(M 

In addition to greater test realism, random vibration testing has been shown 
to be more effective than sinusoidal vibration for screening. Workmanship errors such 
as poorly soldered electri- 
cal connections, loose 
screws, scraps of wire, 
etc. are quickly identi- 
fied. 



Figure 1 of the Standard 
shows that vibration and 
equipment power are to be 
switched on and off, while 
temperature and supply 
voltage are to be varied. 

Sufficient moisture should 
be provided to give visible 
condensation, frosting or 
freezing depending upon 
temperature . 

TEST CONDITIONS/LEVELS 

Hopefully, the combi- 
nations of environmental 
test conditions and levels 
applied under this Stan- 
dard will fairly represent 
field and use conditions. 

Section b.} lists prefer- 
ences: (1) test conditions 
and levels should be based 
upon measured stress and 
levels at proposed equip- 
ment locations during 
typical missions. Failing 
that, (2) estimates may be 
based upon measured envi- 
ronmental data from similar 
appl ications . Only (3) 
when stresses are not spe- 
cified, or when measured 
stresses or estimates are 

not available, should test personnel use the stress types and levels found in Appendix B 
of the Standard. The random vibration aspects of Appendix B will be discussed later. 


— — —Equipment oH (can be operated tf required) 
——Equipment operated in accordance with duty cycle 

t. Time for chamber to reach stabilisation at higher temperature 
I. Time of equipment operation at higher temperature 
: Optional Hot Soak and hot start-up checkout 

FIGURE 1 from MIL-STD-781C 
Sample Environmental Test Cycle 


^ Applies to 
';emp< 
ycle 


j temperature 

J c > 




COMBINED ENVIRONMENTAL CONDITIONS/STRESSES 


1 


Further, Section 4.3 requires that the stresses of varying voltage, vibration, 
equipment on/off and variations of temperature are to be combined in the same chamber 
per Figure 1 (on the previous page) and Table I (of which only the aircraft require- 
ments are reproduced below). 

Table 1. Summary of Combined Environmental Test Condition Requirements 



OPTIONAL NON-STATI STI CAL SEQUENTIAL TEST 

Section 4.4 of the Standard , however, offers options of sequential vibration, tem- 
perature cycling and moisture, j_f approved by the procuring agency. Option I includes 
10 minutes of single-axis random vibration per Figure 2 (see next page of this paper) 
with power applied. (The ordinate should be acceleration spectral density.) Option II 
permits, instead, "broadband/complex" vibration; this term appears to have been included 
to permit the use of certain recent mechanical and pneumatic shakers whose spectra are 
collections of lines (as opposed to the continuous spectra attainable with electromag- 
netic and el ectrohyd raul i c shakers to meet Figure 2 for example.) 
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RANDOM VIBRATION NEEDED 


,04g J /HZ 


To quote Section 
4. 3.2.2 from the Standard 
concerning Vibration Stress , 

" The intent is to 

produce, in the equipment 
on test, a vibration res- 
ponse with a character, mag- 
nitude, frequency range and 
duration similar to that 
produced by the field ser- 
vice environment and 
mission profi le 

Certain modes of vibra- 
tion-induced failure, found 
in flight (particularly 
aboard jet aircraft, mis- 
siles and external weapons), 
also in random vibration 
testing, seldom if ever 
appear with sinusoidal vib- 
ration testing. Combining 
random vibration with other 
stresses, as in -781C. gives 

highly encouraging test results. Laboratory failure modes tend to closely resemble in- 
flight failure modes. Laboratory mean times between failure (MTBF's) tend to closely 
approximate in-service MTBF's. Among the kinds of failure that appear are those due to 
poorly-soldered connections, due to loose scraps of wire, due to loose screws, etc. A 
major value of random vibration testing is screening against such defects. 



FREOUENCY - Hi 


FIGURE 2 

Random Vibration Acceptance Test 


However, during 1975 and 1976 several drafts of -781C drew strong negative respon- 
ses from some contractor firms which had been happily performing reliability demonstra- 
tion tests according to the earlier issues. The need to invest in new shaker systems 
unfortunately requires vast capital outlays, more highly trained and motivated test 
personnel and other expenses. Hence those firms have been greatly interested in low- 

cost mechanical and pneumatic shakers^’ 


Some contractors remember unhappy experiences with early random vibration testing 
systems which were expensive to purchase, difficult to operate and highly unreliable. 
Fortunately, certain shaker system improvements 1970-1976 contributed greatly to ease 
of operation and to shaker system reliability. Chief among these were: 

More rugged electromagnetic shakers, 

Solid-state power amplifiers, and 
Solid-state, digital controls. 

Still, for many firms and agencies, random vibration testing. on electromagnetic or 
electrohydraul ic shakers is formidably expensive. 


Environmental testing laboratories can reproduce in-service vibration using mag- 
netic tape recorded in service. This practice is used extensively in road simulation 
tests of autos and trucks. However, aircraft and missile environmental testing labora- 
tories generally synthes i ze random vibration using noise generators, limiters, 
equalizers and a host of other analog or digital controls. 

TYPES OF SHAKERS 


The low frequency (generally below 60 Hz) sinusoidal vibration needed for fixed 
ground equipment and for shipboard equipment has long been supplied by inexpensive, 
reliable, easy- to-operate mechanical shakers. Higher frequency sinusoidal vibration 
tests for ground vehicles, helicopters and turboprop aircraft, as well as random vibra- 
tion tests for fighters, jet transports, jet bombers and for air-launched weapons and 
external stores, necessitate expensive and complex shakers and controls. Electromag- 
netic shakers are most widely used in the frequency range (to 2,000 Hz) needed for 
avionics and weapons tests. However, electrohydraul i c shakers are increasingly favored 
for low-frequency, long-stroke, high-force applications. Multiple pneumatic shakers as 
well as special mechanical shakers are contending for random vibration tests under 



-781C. Their acceptance depends upon detailed interpretation of vibration test require- 
ments and is not assured at the present writing. These types of shakers were briefly 

reviewed in 1977^’ 

TEST CONDITIONS PER APPENDIX B 

To rephrase instructions found under Section 4.3 of the Standard, the stress types 
and levels found in Appendix B are only to be used if measured or estimated types and 
levels are not available. With that understood, let us examine key vibration points of 
Appendix B. 

Section 40 requires that every significant life-cycle event be considered, includ- 
ing transport i ng , handling, installation, checkout and each tactical mission. Environ- 
mental conditions are to be combined. 

Section 50 is divided into 6 categories according to equipment usage. Section 50.1 
provides combined environmental stresses for fixed ground equipment. Mechanical shakers 
readily generate this sinusoidal vibration. Random vibration tests should be added for 
screening against poorly soldered connections and other workmanship flaws, in the 
writer's opinion. 


Section 50.2 provides combined environmental stresses for mobile ground equipment, 
vehicle mounted. Sinusoidal vibra- 
tion is to be swept 5~500 Hz. fuselage 

Electromagnetic shakers will most 
likely be used, although the require- 
ment for one inch stroke at low test % 

frequencies may cause difficulty. w 0 ^ / 

Some random vibration could easily £,2 y s\ |\. / 

be added via the same shaker. o'® +4dB/octave\ / ! 1 

5 I T'"’ I | \ 

Section 50.3 provides combined s J W) _ / I j 

stresses for shipboard equipment. | j * f 

0.02 inch stroke is to be held while < 

sweeping 4-33-4 Hz. Mechanical sha- j 

kers readily generate this sinusoidal | I j 

vibration. Again, this writer urges 1 1 1 

random vibration for screening. 15 178 300 1000 2000 

FREQUENCY <Hzl 

Section 50.4 provides combined 
environments for jet aircraft equip- 
ment, including random vibration. 

When design flight envelopes and 

specifically designated flight mis- wing & fin tip 

sion profiles are not available, the 
Standard provides Tables B- 1 1 , B- I I I 
and B- 1 V (not shown here) for devel- 
oping environmental profiles. Lack- .« w 

ing specific mission times, 1 hour 40 !\ 

minutes is to be used for fighter | N. -3dB/Octave 

aircraft and 6.5 hours for transport/ J's I / 

cargo aircraft. An extra benefit J \. / 

from the random vibration is effect- s | I n. 

ive screening. | J \. 

Section 50.4.3.2.1 requires I 

random vibration in one axis. MIL- 

STD-810 is referenced for tolerances L— • J — 

and for mounting. The acceleration 5 100 2000 

spectral density of applied random frequency ih^i 

vibration, as measured on the test 

fixture at the mounting points of the . FIGURE B-4^ 

test item, is to be obtained from Aircraft Random Vibration Test Envelope 

Figure B-4. Ordinate value W 0 is to 

be obtained from Table B-V, Figure B-5 (dynamic pressure units should be 1 b/ ft 2 ) and 


FREQUENCY (Hz) 

FIGURE b-4 

et Aircraft Random Vibration Test Envelope 


the general notes which follow. 


3 decibels less than W c 




Aerodynamic Induced Vibration 


W 0 = K(q) 2 

q = Dynamic Pressure (when q > 1200 psf use 1200) (See 50.4.3.2.2a) 

M 1 = W Q -3 dB 

(FIGURE B-2 for Spectrum Shape) 

K 

Equipment location 

.67 x 10" 8 

Equipment attached to structure adjacent to external surfaces that are 
smooth, free from discontinuities. 

.34 x 10' 8 

Cockpit equipment and equipment in compartments and on shelves adjacent 
to external surfaces that are smooth, free from discontinuities. 

3.5 x 10' 8 

Equipment attached to structure adjacent to or immediatley aft of sur- 
faces having discontinuities (that is, cavities, chins, blade antennas, 
and so forth) 

1.75 x 10' 8 

Equipment in compartments adjacent to or immediately aft of surfaces 
having discontinuities (that is, cavities, chins, speed brakes, and so 
forth) 


SPECIAL CASE CONDITIONS 


Fighter Bomber 

Condition equipment location 

Take off/attached to or in compartments adjacent to structure directly 
exposed to engine exhaust Aft of engine exhaust plane (I minute) 
Cruise/(same as above) 

Take off/in engine compartment or adjacent to engine Forward of engine 
exhaust plane ( 1 minute) 

Cruise/{same as above) 

Take off, landing, maneuvers/wing and fin tips* decel leration (speed 
brake) (1 minute) 

High q (> 1000 psf)/wing & fin tips* 

Cruise/wing & fin tips* 

Take off/all other locations (1 minute) 

Cargo/Tra ns port 

Condition/equipment location 

Take off/fuselage mounted 
Take off/internal 

Take off/wing-Aft of engine exhaust** 

All/wing tip and fin tip*** 


* Use wing and fin tip spectrum - FIGURE B-4 

** Excludes Upper Surface Blown (USB) and Externally Blown Flap (EBF) 

*** Take off, landing, plus 10 percent of cruise time 


.7 

.175 

.1 

.025 

.1 

.02 

.01 

.002 


.01 

.005 

.05 

.01 


TABLE B-V 

Jet Aircraft - Random Vibration Test 





FIGURE B-5 

Dynamic Pressure (q) as Function of Mach Number and Altitude 
50.4.3.2.2 General notes . 

a. Determination of mission profile vibration levels. The vibration level for each 
phase of the profile will be determined from TABLE B- V. A maximum of four W '$ 
will be determined by: (1) take off; (2) q ; (3) q ■ ; and (4) q . Where q 
is the maximum aerodynamic pressure encountered during “he mission, usual ly at low 
altitude, high speed dash: q . is the aerodynamic pressure associated with the 
flight phase that will generauaa W above .001 g /Hz, normally a cruise condition. 

The fourth vibration level shall be°determined by combining the vibration levels 
calculated for each of the other phases (climb, dive, combat, and so forth) having 
levels above the minimum (see c. below ). An average q shall be used for each 
flight phase, that is, for a phase such as dive, the arithmetical average of the q 
at the start of the dive plus that at the termination of the dive shall be utilized 

( + q, )/2 = q„ „ . A shall be calculated for each phase and flight 

start term avg o 

phases with W 's above the minimum shall be time weighted averaged to determine the 
fourth vibration level 

b. Cargo aircraft. Unless unusual mission profiles are determined, take off and 
cruise profiles (vibration levels) will be the only required vibration levels 

c. Minimum W Q test level. The minimum W 0 test level shall be .001 g /Hz. If the cal- 
culated test level is less than .001 g 2 /H z, vibration test is not required during 
this portion of mission profile 

d. Option. Maximum W determined may be used throughout test 

e. Gunfire environment. Not considered in this test. Should be considered in envi- 
ronmental qualification test, if applicable ( MI l_ -STD-8 10 ) 

f. Composite vibration profile. Turboprop and jet aircraft usage; when equipments are 
to be installed in both turboprop and jet aircraft, a composite random spectrum 
shall be generated (see FIGURE B-6 for example, composite spectrum) 

g. Wing and fin tip, fuselage equipments. When equipment is to be installed in both 
locations, a composite vibration profile shall be utilized where appropriate 

An example of the construction of a mission test cycle profile is given under 
Section 50. 4. A of the Standard. Assumtions: 

a. Equipment Class 2 in accordance with MIL-E-5400 

b. Equipment installed in air-conditioned compartment 

c. Equipment is ambient cooled (no supplemental cooling) 

D. Equipment is attached to structure adjacent to external surfaces that are 

smooth, free from discontinuities 

e. Altitudes are shown in Figure B-7 (not shown here). The vibration conditions 

for each phase (two flights) are shown in Tables B-XIV and B-XV (next page). 
Phase A, lasting 30 minutes, represents cold day operations, no vibration. Phase B 
contains one minute vibration to represent cold-day takeoff and 6 minutes to represent 
climb to 30,000 feet. Phase C represents 23 minutes Mach 1.0 cruise, two minutes Mach 
1.0 dive to 10,000 feet, 5 minutes Mach 1.0 intercept maneuvers, 13 minutes climb to 


Si 

SK 

m 

m 





TABLE B-XIV 

Calculations for Vibration Test Levels 


Test phase 

Altitude 

(feet) 

Mach 

number 

g Wo 

(9 / Hz -) 

Duration 

(minutes) 

A 

Ground Operation 

0 

0 

0 

30 

B 

Take Off 

0 

0 

.002 

1 

B 

Cl imb 

1-30 K 

.6 

.0006* 

6 

C 

Cruise 

30 K 

1.0 

.0012 

23 

C 

Dive 

30-10 K 

1.0 

.0035** 

2 

c 

Intercept 

10 K 

1.0 

.0067 

5 

c 

Cl imb 

10-40 K 

1 . 0-.6 

. 002 ** 

13 

c 

Cruise 

40 K 

.6 

.00006* 

35 

D 

Descent 

40-0 K 


.0004* 

15 


* In accordance with 50.4.3.2.2c 
** Mix vibration levels = ( ,0035) 

for major flight phases. 


value below minimum therefore no vibration is required. 

_{ 2 ) + ( .002) 13 determined by time weighting vibration’levels 


40,000 feet and 35 minutes Mach 0.6 cruise 

to base. Phase D represents 15 minutes _ TABLE B-XV 

idle descent to a hot-day landing. Phases Final Vibration Test Conditions for Example 

E through J represent similar operations 

starting with a hot and ending with a cold Vibration Level Duration 

day. Vibration testing is not required Test Phase (9 ^ z ) (minutes ) 

when levels are less than Q.001c? 2 /Hz. A, F 0 30 

Levels for the "dive" and the "climb" 

phases have been time averaged. These G 1 

manipulations simplify conducting tests. e > q q 

Section 50.5 provides combined envir- c > H .0012 23 

onments for turboprop aircraft and for c H 

helicopter fusel age- 1 ocated equipment. ' ’ 

Gunf i re- i nduced vibration should be con- C, H .0022* 15 

sidered when an equipment is destined for 

attack helicopters; Method 519.2 of MIL- C ’ H 0 35 

STD-810 should be consulted. 0, I 0 15 

Section 50.5-3 calls for swept-sine E 0 

vibration, one sweep in 12.5 minutes (10 

minutes for Army). The writer is told of * Mix vibration level. 

/ O \ 

plans^ to change Figure B-6 (which 
accordingly is not reproduced here) and 

some of the text under 50.5-3- Random vibration is not contemplated under 50.5.3. 

The writer feels this to be a mistake; some random vibration, at least for workmanship 
screening, should be added. 


Test Phase 

Vibration Level 
(gVHz) 

Duration 

(minutes) 

A, F 

0 

30 

8, G 

.002 

1 

B, G 

0 

0 

C, H 

.0012 

23 

C, H 

.0067 

5 

C, H 

.0022* 

15 

C, H 

0 

35 

D, I 

0 

15 

E 

0 

- 


* Mix vibration level . 


Section 50,6 provides combined environments for air-launched weapons and assem- 

(Q) 

bled external stores. MTL-STD-1670 K is referenced for environmental criteria and 
guidelines. Unfortunately, while -781C was being developed, -1670 was replaced by 

-1670A ^ ^ . The latter is somewhat condensed; it does not give spectra, but rather 
refers users to MIL-STD-810 . 


CONCLUSION 

Flight environments have changed greatly since World War II. But the vibration 
portions of long-time reliability-demonstration tests have stuck with pre-War tech- 
nology. This writer, for one, is delighted that broad-spectrum random vibration (and 
acoustic) testing is at last replacing the single-frequency tests of MIL-STD-781, -A 
and -B. An important message may be seen by reading -781C i -810C and -1670: the mili- 
tary services want greater realism in vibration testing (reliability demonstration 
as well as qualification). They also want equipments screened for workmanship flaws. 
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INTRODUCTION 

We were asked to set up a comprehensive quality control system for a product that was pushing the state of 
the art and causing a serious low yield by our subcontractor. The situation facing us was this: an unknown 
product, an enigmatic and sensitive series of processes, a high-production condition, and a high cost per unit. 
Since the typical aerospace condition is one of low production rates, the product challenged our customary 
quality control methods. 

This paper recounts how we handled the new product and the unusual conditions. It can provide the basis 
of an approach to someone confronted with a similar problem. As a context for the rest of this presentation, a 
brief description of the product and process follows: 

The thermal protection system for the Space Shuttle orbiter consists of a high-purity silica-fiber insulation 
built as a tile and designed to protect the vehicle structure against temperatures of 2300°F during atmospheric 
reentry. Each Shuttle vehicle will require approximately 34,000 tiles in thousands of different configurations 
capable of withstanding repeated heating and cooling, plus extreme acoustic environments, for up to 100 flights 
without significant refurbishment or replacement between flights. 

The basic raw material is a 99.7-percent pure amorphous silica fiber. It is cast, dried, and sintered to form 
lightweight silica blocks, which are then machined to the required dimc'nsions. A coating of borosilieate glass is 
applied and sintered at temperatures of 2100 to 2200°F to fuse the glass coating. Treated with a waterproofing 
agent to prevent absorbing extra weight from rain or humidity, the tiles are then bonded to a pad of Nomex felt 
(used as a strain stress isolator) for installation on the Shuttle orbiter. 

PRE-PRODUCTION QUALITY INVOLVEMENT 

The need for early involvement in process planning cannot be overemphasized. It is here the quality control 
professional establishes working relationships with his manufacturing and engineering counterparts. This was 
especially true when ceramic processing technology was introduced into aerospace. It was realized that the 
quality control practitioner had a certain expertise in process control. So instead of being driven, as is usually the 
case, the quality engineer was part of an integrated approach from the beginning. 

Process Control Emphasized 

The strategy used for this critical and creative phase of the process was to concentrate on control of the 
process rather than inspection of the product. 

The key then was understanding the process, for which a detailed flow chart of the manufacturing processes 
were prepared. Figure 1 presents a section of this manufacturing/processing flow chart. The chart depicts each 
operation performed, quality characteristics to be inspected, controls o'n raw materials, applicable inspection and 
test instructions, sampling plans, and control charts. This technique enabled the quality/process control engineer 
to locate the inspection activities to the best advantage while minimizing the impact on the upstream 
manufacturing activity. In addition, it allowed him to keep his finger on the pulse of a dynamic process and 
provided him with the flexibility of changing inspection stations and operator verification points as data were 
gathered during the process. The chart stresses maximum use of operator (Manufacturing Department) 
verification points while still maintaining a disciplined quality system with maximum flexibility and operator 
feedback. 

The operator verification approach stems from two considerations. First, involving the technician in the 
inspection of his work raises his consciousness of the product’s quality. Not only is he solely accountable for 
delivery of the product on time, bur also he is made aware of his role in the quality of the product when it is 
delivered to the subsequent manufacturing stations. Second, involving the manufacturing technician in the 
inspection tasks is more cost effective. Inspection tasks delegated to the operator verification system encompass 
items that can be verified visually. Tasks that are critical or require special knowledge or unique inspection 
devices are not delegated to manufacturing. 
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Figure /. Manufacturing and inspection flow chart. 


Thus, what started as a process logic flow to understand the process ultimately grew to 
document, which proved to be invaluable in subsequent planning. 


quality planning 


Although the flow chart enables the quality/process control engineer to cover the “big picture” effectively, 
another means of collecting the details of the quality assurance process is required. Figure 2 provides a method 
for defining product characteristics: what, when, how, how many, etc. In addition, when incorporated into the 
manufacturing/processing flow diagram, the inspection definition provides an easy and accurate method for 
predicting manpower needs. 

Traceability 

Coupled with the need for critical process control was the need to maintain traceability. Key elements of 
the process, including subcontractor billet numbers, coating slurries, and waterproofing agent lot numbers, had to 
be recorded against each tile and tile lots produced. The total number of characteristics that require traceability 














equaled 32. The typical two-way street of traceability raw material to installation to raw material had to be 
economically maintained. Since the traditional methods of operation relative to procedures and documentation 
lifficult to sever, existing systems were tailored to the needs dictated by the process. Volume alone would 


Pooling the resources and talents of various organizations within the quality/manufacturing management 
structure made it possible to simplify the manufacturing order (the record that accompanies the part to be 
fabricated) while traceability through the use of the process control record (Figure 3) was maintained. The 
process control records (PCR’s), serialized by date, contain all information pertinent to the process; e.g., slurry 
roll time, gun delivery rates, batch numbers, operator numbers, and product serial numbers. The PCR serial 
number is then cross-referenced on each manufacturing order against each specific operation performed. Thus all 
data are entered on the PCR, which enables the processing and traceability data to be easily retrieved by either 
serial number or lot number. Data relative to the process were maintained by the operator at each processing 
station. A job classically done by inspection was thus delegated to the Manufacturing Department, with minimal 
surveillance by inspection. 
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Systematic Approach 


To summarize, tire systematic approach used during the preproduction phase enabled Quality Assurance to 
move smoothly into the production phase and avoid the confusion of developing a quality system in parallel 
with initiating a new product or process. 

The flow chart facilitated communication among Quality Assurance, Engineering, and Manufacturing and 
insured satisfaction of common goals. Since the flow chart was developed in the Quality Assurance function and 
later adopted by the other two functions, Quality Assurance took the lead in planning during the preproduction 
phase. This in itself can be considered a “first." 

PRODUCTION QUALITY INVOLVEMENT 

Since the strategy for this process was to stress process control rather than product inspection, timely 
feedback of information was critical. Feedback mechanisms, in the form of control charts, were designed around 
each inspection station to provide the operator and management with real time data. Characteristics to be 
inspected hacl been carefully set forth on the manufacturing process flow chart and this, coupled with the 
process control records, provided Quality Assurance with the tools to glean the needed information for control 
charts and to perform meaningful analyses. 

The statistical techniques employed are those of the classical textbooks on quality engineering. Because 
fabrication of these tiles requires multiple steps and involves numerous variables, a fundamental statistical 
method employed was analysis of variance (ANOVA). Potential interaction effects usually involve several 
independent variables and require more sophisticated ANOVA models. However, when the analysis of variance 
revealed that no interactions existed, the data could be more effectively communicated by control chart 
techniques. 

Both X bar and P charts have proved effective in feeding back the information to the responsible 
departments. When feasible, the chart is plotted by the manufacturing technician who is performing and 
verifying the work being monitored. This involvement eliminates the criticisms of some technicians, who call the 
control chart concept too remote or impractical. It gives the technician immediate information relative to his 
responsibility in a format easy to interpret. 

SUMMARY 

Our experience has demonstrated that Quality Assurance not only can become an integral part of a team 
composed of Manufacturing and Engineering, but also can be the integrating force. We were successful in this 
role because we were systematic and comprehensive in our approach to the quality matters and because wc 
became involved early in the process. Another factor is that Quality Assurance defined its strategy: wc claimed 
responsibility and accountability for developing information feedback systems so that those responsible for 
action had valid and timely data. The confirmation of this careful planning is that we have experienced a yield 
rate of over 90 percent and have met our early budget commitments. 

We have also confirmed that the classical Quality Control concepts work. Control charts, Pareto analysis, 
analysis of variances and the other traditional textbook tools of the quality engineer arc indeed his tools: perhaps 
lie alone of the team members from Manufacturing and Engineering has the expertise in the collection, 
processing, and interpretation of the data. In 1931, W. A. Shcwhart wrote the first textbook devoted to quality 
control, He entitled it liconomic Control of Quality of Manufacturing Product and his theme was that quality 
control does contribute meaningfully to the economics of manufacturing processes. It has been our experience 
that Shewhart is still timely. 
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TEST: TOLERANCES & MEASURING - 
BOTH VITALLY WEAK IN Q.C. 

Dorian Shainin, Consultant 
Manchester, Ct. 060l|.0 


In the control of quality, in its assurance, and in auditing quality 
we MEASURE against pertinent STANDARDS. Even if the quality char- 
acteristic happens to be a cosmetic, visual one, we should still have 
a reference, a standard for deciding we are IN or OUT of specifi- 
cation. Judgment can vary more if the standard is a mental, opinion 
(of workmanship appearance) one; so most companies keep a physical 
"limit sample" to serve as a borderline acceptable reference for each 
such characteristic of some importance. 

Statistical quality control developments in manufacturing process 
control, and in acceptance sampling have been substantial over the 
years. Surprisingly, similar efforts were not expended in statistically 
evaluating the quality of the "numbers" for the two basic parameters 
capitalized in our opening sentence: the definition of quality and the 
validity of the measurements of the product. Apparently, it has been 
too easy for managers to assume that tolerances which survive with time 
and extensive product use must be close to correct; and to assume that 
periodic reinspection of gages and instruments against reference masters 
under metrology room conditions represents all, or about the best, a 
company can do. 

A non-statistic ally, usually arbitrarily, selected tolerance “almost al- 
ways adds a non-productive cost. And we have found that many measuring 
devices change in their calibration when they leave the metrology room 
and are used under operational environments. That is not surprising. 
What is unexpected is that usually gages that are the same design do 
not change in their settings the same amount. Let’s define such dif- 
ferences as gage-to-gage bias. 

I. MEASURING 


Ironically, those differences often DISAPPEAR when the gage’s original 
settings are rechecked back under metrology conditions. Accordingly, 
management of this problem needs information about the mean and the 
dispersion (precision) of the measuring instruments under operational 
environments, instead of under calibration environments. One of these 
environments is the true, but unknown, product to "similar" product 
variation, AP, which sometimes disturbs the uniformity of the bias. 

For a remarkably effective plotting strategy, enter the results from 
two similar gages for at least 30 different units of product, on a 
two -coordinate graph with equal scales, one for gage #1, the other for 
gage #2. Each plotted point represents the pair of readings on a single 
unit. By eye, encompass the points with an ellipse, touching the 
boundary points, or excluding the one "most extreme" point. That el- 
lipse becomes a rough, but not -too -had estimate of a 95 percent proba- 
bility contour (excluding 5 percent of 30 points, which would be 1-1/2 
points) . 

Draw a straight line from the origin at a 1+5 degree angle to a coor- 
dinate axis. It represents the line on which all equal readings from 
the two gages will lie. If the major axis of the ellipse lies on that 
1+5 degree line, the two gages agree, on the average, A rather common 
surprise is the discovery of a displacement from, and sometimes an 
angular relationship of the major axis to the 1+5 degree line. That 
"bias” between the two gages reveals the nature of the systematic error 
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introduced, on the job, unexpected by the fact that both were released 
by metrology control as being alike. 

In addition to possibly having a displaced mean result, a gage has a 
certain degree of lack of repeatability, called its imprecision. The 
30 perpendicular distances from the points to the degree line repre- 
sent a s ample of 30 estimates of just the measuring error from both 
gages. No product variation contributes to those perpendicular dis- 
tances, because product variation (AP) only causes the points to move 
parallel to the l\S degree line, toward or away from the origin. 

Thus the small width dimension of the ellipse, which is perpendicular 
to the I+5 degree line, can be used as an estimate of AM, measuring 
error. This estimate improves as the true AM’s for each of the two 
gages approach being equal to each other. 

At 90 degrees to this dimension, mark off the large dimension of the 
ellipse as a parallel length, along the degree line. With that 
length being the hypotenuse of a degree right triangle, 0,707 times 
it will be the length of one of the shorter sides, equal to the chance 
combination of AP and AM variation. The variances of those two inde- 
pendent estimates add statistically. Thus the square of that short 
side length equals AP 2 + AM 2 , The solution from that equation for AP 
provides a statistical estimate of the true AP from the sample of 30 
points. Here we are estimating the true product variation, with all 
the AM gage error removed, regardless of the imprecision of the gage. 

The ability of this plotting strategy to furnish independent estimates 
of gage bias, gage precision, and true product variation has led to a 
growing number of useful adaptive applications: 

- qualifying the discriminatory precision of the measuring method 
for the measuring task: e.g. when the ratio of AM/AP is 1/6 or^less, 
the method will give single readings with an error of 1/3 6(+1 .1$) or 
better, 

- comparing the relative abilities of two or more technicians or 
inspectors to perform their evaluation tasks alike. 

- vendor versus buyer ability to agree on the evaluation of a 
quality characteristic. 

-laboratory to laboratory differences. 

- comparing two or more different measuring equipments, or methods. 

- even two or more different formulae have been compared statis- 
tically for calculating a performance parameter from a number of input 
measurements . 

The reader is expected to extend this list. 

II. STANDARDS 


A quite different plotting strategy has statistically evaluated the 
quantitative tolerances for raw material, manufacturing processes, parts 
subassemblies, etc., intended to result in certain acceptable output, or 
final product, quality characteristics. A statistician would name the 
input tolerance as defining the independent variable; the output require 
ment as the dependent variable. 

To evaluate the validity of the limits for any independent character- 
istic, measure a series of 30 inputs on typical (random) parts, and tag 
each with its value, later to be plotted on the x-axis of a graph. Who 
ever originally specified a tolerance for that x felt it might, or would 
have an influence on at least one output, causing that dependent vari- 
able y to vary. The 30 tags will permit you to evaluate the validity of 
that influence, IF IT EXISTS AT ALL! When those parts are assembled. 



combined with others to make a subassembly, or a finished product, the 
corresponding 30 y output measurements are entered on each tag moved 
from the part to the assembly containing it. 

The graph displays the 30 (x, y) points. Again, by eye, encompass the 
points with a 95 percent contour ellipse, or a circle if that be the 
case, and interpret the outcome. 

When the axes of the ellipse are parallel to the x, y coordinate axes, 
or when you have a circle, a change in x produces no change, on the 
average, in y. The selection of the interval size for plotting the 
x-axis and for the y-axis have a bearing upon this observation. Use 
this "rule of thumb": Be sure that the scale interval you select for 
the 30 points shows 7 or more intervals on each axis. Too coarse a 
scale will desensitize the information sought. 

But when the axes of the ellipse are tilted, an average influence of x 
upon y is indicated. In that case draw, again by eye, a major axis 
of the ellipse, with 1 5 of the points above and below it, called the 
regression line or curve in the accompanying illustration. Also draw 
a pair of parallel, 95 percent boundary lines, equidistant from the 
regression line. 

Your customer, if he is like most customers, has clearly established 
his requirement for y in terms of a tolerance. If he has specified 
both a maximum and a minimum, extend a horizontal line from each to the 
upper and to the lower boundary lines, respectively. Prom those inter- 
section points, as shown in the illustration, drop lines perpendicular 
to the x-axis. You can see that if production, in the future, complies 
with that new, indicated realistic manuf acturing tolerance for x, vir- 
tually all the future points might be expected to lie within a paral- 
lelogram, and all between the customer’s desired limits, 

A most cost-effective characteristic (of this procedure for maintaining 
traceability of each of the 30 x’s to the y of the assembly or system 
they enter) is that the identical 30 assemblies can show a possible re- 
lationship to 30 X'l ’s, 30 xp's* * 3 * 3 , ©tc. Just use more tags. 

With n different input tolerances, expect this evaluation with the 
single set of 30 assemblies to follow the Pareto principle: One, or 
very few of the x's may be found to be vital; the great majority will 
have trivial or no influence upon y. 

To distinguish the vital from the trivial, do not compare their plotted 
slopes. Bach x has its own scale, different from the others. Scale 
selection influences the apparent slopes. 

Instead compare the Ay, for a fixed value of x, between the boundary 
lines. All the y axes on the different x graphs should be plotted with 
the same scale for the one set of 30 output measurements. To identify 
the x that has the greatest influence upon the required y output, notice 
that its chart will exhibit" the narrowest Ay for a fixed value of x 
(between the two arrow heads in the illustration). 

Ironically this evaluation procedure, for a number of x’s suspected to 
have an influence upon an important customer y, often reveals that all 
the x’s have virtually no significant influence. The ellipse axes 
parallel the x,y axes. The vital x remains unsuspected then, or oven 
not specified. But please remember: No y variation occurs without 
some cause. You become a real manager of quality when you persist in 
your search for the most vital x. You improve your company’s ratio of 
quality to cost, which can be called product "value" to the customer, 
when you open up unnecessarily tight tolerances, and graphically esti- 
mate, then confirm with new x,y data, the realistic tolerances for the 
important x’s. 

The remaining step relates to important visual, rather than quantitative 
standards. An evaluation of the ability of important customers to pass 


a rank order discrimination test serves well here* A minimum sample of 
six items can work with 95 percent confidence. Three parts are repre- 
sentative of the present limit sample, described in our opening para- 
graph. The other three are examples of a new, less rigid limit sample 
that would save a computed number of production dollars - successfully 
with no quality penalty if customers cannot tell the two standards 
apart. 

The test for typical, important customers: "Please rank these six items 
as 1 to 6 from best to worst for this visual characteristic we are 
pointing to." If they succeed in placing the three tight ones in the 
top ranks, they can see the difference. But if they fail to separate 
the tight three from the looser three, our present limit sample is 
failing to achieve anything with its higher cost. 

Since there are 20 different ranked order arrangements possible of 3 
tight and 3 loose items, only 5 percent of the time will the tight ones 
rise to the top ranks if each evaluator is only guessing. If he really 
cannot discriminate, 95 percent of the time we shall be protected 
against keeping the expensive, unnecessary tight standard. If he 
clearly can discriminate, then 100 percent of the time we shall maintain 
the tighter limit sample. 

•This paper has suggested that you, too, as many companies have now done, 
extend the rather simple statistical strategies of quality control to 
measurement evaluation with 45 degree comparison graphs and to estab- 
lishing realistic standards for important characteristics by objective 
tests : 

- does a current range of x influence y as intended? 

- can the customer tell the difference between one qualitative 
standard for y and another less demanding? 

You owe it to your professional wellbeing to serve your company in con- 
trolling and assuring quality objectively. 
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ESTIMATING AUTOMOTIVE RELIABILITY 
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INTRODUCTION 

Reliability from a theoretical standpoint is a reasonably well understood disci- 
pline that has experienced considerable research effort over the last twenty years. 
Reliability is usually defined as the probability that a system will perform its in- 
tended function adequately for a specified interval of time when operating under stated 
environmental conditions. Any system has an inherent reliability level due to the pa- 
rameters of the design. That is, the reliability level is established at the design 
phase, and subsequent testing and production will not raise the reliability without 
a basic design change. 

Implementing a reliability program in a consumer industry always presents some 
unique problems. First of all, reliability is a probabilistic concept, and its meas- 
urement must be approached from this standpoint. However, probabilistic concepts and 
statistical estimation are, at best, poorly understood by many in the design activity. 
Thus, there is a tendency to use such terms as percent failures or failures per hundred 
based on simple calculations to represent reliability, and these terms are frequently 
volume dependent. 

Also, even if one addresses the probabilistic nature of reliability from a ration- 
al standpoint, other problems will be encountered. With commercial products, the en- 
vironmental conditions must usually be taken as those encountered in use. This is in- 
deed a problem since commercial products (and particularly vehicles) will experience 
wide variability in use. Since just a few failures can promote a recall campaign of 
extremely costly consequences, the tendency is to over design to provide a wide margin 
of safety particularly for critical items. 

Product testing is usually not conducive to reliability estimation for two reasons 

(1) The testing is not extensive enough particularly for highly reliable products, and 

(2) Any design deficiencies that show up during testing are corrected. So, the basic 
design is continually improved throughout testing and only stabilizes when it is 
released to production. 

Eventually, the products reach the consumer market, experience field use, and some 
occasionally do fail. Typically products have a warranty period, and failures during 
this warranty period are reported back to the manufacturer by way of an information 
system. In order to measure, compare, and improve product reliability an estimation 
procedure is needed based on the particulars of this situation. That is, estimation is 
frequently situation oriented in that standard techniques must be tailored somewhat to 
the situation at hand. 

This paper focuses on a particular product and industry - the automobile. This 
product has a significant impact on the consumer market and considerable effort is ex- 
pended in design improvements. Estimated reliability is used as one input to decisions 
for re-design and engineering changes. The information source most frequently used for 
predicting vehicle reliability is quite similar from manufacturer to manufacturer and 
is essentially a warranty (or failure) reporting system. The next section will briefly 
discuss some reliability prediction methods in common use based on warranty data. 
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BRIEF REVIEW OF COMMONLY USED METHODS 


Vehicle reliability determination is not a new problem and one may rightfully ask, 
"What methods are presently being employed?" Review of presently used methods also 
serves to provide familarity with the particulars of the problem situation. The meth- 
ods reviewed herein have been developed by various individuals working in different 
automobile companies over a number of years. It is extremely difficult to trace; orig- 
inal authorship of these methods. Thus, with apologies, authorship is not acknowledged. 
Also, it must be recognized that this review may not be complete, since it is difficult 
to extract company information on this subject. The review is essentially based on our 
experience with engineers from various automobile companies. 

One simple and frequently used method is to plot the failure odometer readings on 
Weibull paper. Another method is to sum failure odometer readings of reported failures 
and divide by the cumulative number of failures. The problem with these methods is 
that the unfailed or suspended mileage is ignored, and the calculations are not really 
indicative of the true reliability. Some also use failures per 100 vehicles sold (or 
produced) and term this R/100. This is roughly equivalent to the "failures as a per- 
cent of sales" method to be explained in more detail in the next section. 

Failures as a Percent of Sales (or Production) 

This is a simple and supposedly easily understood method used for tracking warranty 
failures and hence warranty costs. For a particular vehicle line the ratio of total 
failures, to date, to total sales (or production) is calculated as a percentage. This 
percentage is plotted over time to indicate trends and to predict final warranty costs. 
So, for example, if sales, to date, was 300,000 vehicles with 280 reported failures, 
the resulting percentage would obviously be 0.09. 

Recognize that variability in the volume of monthly sales affects this ratio, be- 
cause the vehicles sold in the early months have a longer time to accumulate failures 
than those sold in recent months. However, the thought may be that sales are relatively 
stable over time, and this ratio allows a convenient tracking method for comparison, by 
month, to indicate improvements and also to predict final warranty expenses. But sales 
do vary from month to month, and also, this method has one other subtle fallacy that 
will now be explained by way of a simple example. 

Basically the problem results from, first of all, the change in the size of the 
warranty population due to production, and secondly, the losses from the warranty popu- 
lation due to the limited length of the warranty period. Let us take a very simple and 
somewhat hypothetical example to illustrate this phenomenon. It will be assumed that 
exactly 100 vehicles per month are produced and all sold precisely on the first day of 
the month. It will also be assumed that each vehicle is driven exactly the same number 
of miles per month, and the failure rate is constant such that 100 vehicles will gener- 
ate one failure per month, on the average. The period of production will be taken as 
four months and an eight month warranty period will be used. Although this is very hy- 
pothetical it serves for illustration. Based on this situation Table I can be 
generated. 

Tab le I Calculation of Failures as a Percentage of Sales 

No. Vehicles Came 

Month of Production in Ctime Under No. Failures Reported 

Model Year No. of Vehicles Vehicles Warrant y Reported Failures 


1 

100 

100 

100 

1 

1 

1 . 0 % 

2 

1.00 

200 

200 

2 

3 

1.5 

3 

100 

300 

300 

3 

6 

2.0 

4 * 

100 

4 00 

400 

4 

10 

2.5 

5 

0 

400 

400 

4 

14 

3.5 

6 

0 

4 00 

400 

4 

18 

4.5 

7 

0 

400 

400 

4 

22 

5.5 

8 ** 

0 

400 

400 

4 

26 

6.5 

9 

0 

400 

300 

3 

29 

7.25 

10 

0 

400 

200 

2 

31 

7.75 

11 

0 

400 

100 

1 

32 

8.0 

12 

0 

400 

0 

0 




Failure 
as a 

Pore e n t a ^ e*** 


last month of production 

last month of warranty for vehicles produced during first month of mo 
Failures divided by cume vehicles sold or produced. 


* * 

A A A 


i d e 1 year. 



The thing to note in Table I is that the percentage used for tracking and predic- 
tion purposes is calculated as cumulative failures to cumulative production (or sales) 
but the number of vehicles producing the failures in the warranty reporting system are 
those under warranty, and this population changes. A more accurate method would be to 
calculate cume failures to number of vehicles under warranty; however, in "real life" 
this is ignored because the size of this population is not readily known. 

Charting the above calculated percentage over time can lead to some erroneous con- 
clusions when plotted by month, on simple graph paper. For long production periods of 
ten months and 12 month warranty periods one can be led to first conclude an increase in 
warranty claims, and then eventually a tapering off when in fact no change has taken 
place. In the "real life" situation the month to month sales variability and random- 
ness of failures help to mask this underlying trend and make the results seem plausable. 

The Average Exposure Method 

The average exposure method was apparently posed to alleviate some of the problems 
with the previous method. Basically this method calculates percentage of failures as 
before, but plots this percentage using something termed the average number of months 
of exposure. The method can be illustrated with the previous data of Table I. We will 
now assume, however, that sales (or production) are uniform throughout the month. As 
one proceeds through the model year the average age, "timewise," of the warranty popu- 
lation will vary; thus, a quantity termed average exposure is calculated as shown in 
Table II. The failure percentage is then plotted against this average exposure time. 

The motivation behind this method is to obtain a comparable and predictable 
straight line relationship. However, this is not necessarily accomplished, and again 
this method is not amenable to direct reliability estimation. 


Table II Average Exposure Calculation 

Average 

Month of Exposure Time Exposure Time Exposure Percentage 

Model Year for Youngest for Oldest ** (Months) Failures*** 


1 0 

2 0 

3 0 

4 * 0 

5 1 

6 2 

7 3 

8 4 

9 5 

10 6 

11 7 

12 8 


1 

1/2 

1 . 0 % 

2 

1 

1.5 

3 

1 1/2 

2.0 

4 

2 

2.5 

5 

3 

3.5 

6 

4 

4.5 

7 

5 

5.5 

8 

6 

6.5 

8 

6 1/2 

7.25 

8 

7 

7.75 

8 

7 1/2 

8.0 


8 


* Last month of production 

** Which could potentially appear in warranty report 
*** From Table T 

Total Mileage per Interval Method 

Another procedure in use is aimed at estimating the failure rate. Again the basic 
information source for this procedure is the monthly warranty report, or failure report- 
ingdata as available from Peterson, Howell & Heather (a leasing firm in Baltimore, Md.). 
The approach is to estimate the failure rate by mileage interval for a vehicle line. 

A monthly mileage distribution is developed from warranty reports. This is done 
by dividing the recorded odometer reading as reported at the time of failure by the num- 
ber of months since sale of the vehicle. Obviously a monthly mileage ogive can be de- 
veloped by sampling several hundred cases from warranty reports. Thus, for example, if 
15% of the "population" of vehicles accumulated more than 1,000 miles per month, one 
could use this to compute mileage for the (0-1,000) mile interval by using monthly 
sales as indicative of age. 

In order to illustrate, let us assume that we have the end of December warranty 
report, and we want to calculate failure rates. Vehicle production by month is readily 
attainable. Let us say it is as follows: 

Month Vehicle Production 


September 

October 


20,000 

16,000 


and so forth, for the remaining months. The amount of mileage attributed to each inter- 
val by each month's production will be determined by considering the average number of 
months in use. In order to do this, consider Table III below. Here the first 1/2 
month is due to the lag in sales while the last 1/2-month is due to the lag in the re- 
porting of failures through December (using this reasoning it would probably be more 
correct to use zero months usage for cars produced in December; however, this is not 
what is done) . 


Table III 


Average Number of 

Months 

of Use 



Month of 


Months of Use 



Production 

Sept 

. Oct . Nov . 

Dec . 

Total Months Use 

Sept. 

1/2 

1 1 

1/2 

4 

Oct . 


1/2 ] 

1/2 

2 

Nov . 


1/2 

1/2 

1 

Dec . 



1/2 

1/2 


Of the 20,000 vehicles produced in September the vehicles with 2,000 miles or more 
on them, would be the ones that averaged over 500 miles per month. This is obtained by 
noting that September produced vehicles have four months usage. Using the mileage ogive 
(not included) one might find that there is, say, 97% of such vehicles in the "popula- 
tion." Thus, the mileage contributed to the (0-2,000) mile interval by September built 
vehicles would be calculated as 

20,000 x 97% = 19,400 vehicles 

19,400 vehicles x 2,000 mi/vehicle = 38.8 x 1(T mi 
For each month of production, the number of miles contributed to the (0-2,000) mile 
interval is calculated. So, for each mileage interval the total number of miles and 
total failures in the interval are used to calculate a failure rate. 

This method is definitely an improvement over the previous percentage methods; 
however, there are a few problems. First of all, the basis for the mileage ogive was 
not the total vehicle population (or a random sample from it), but instead was the pop- 
ulation of failed vehicles appearing on the warranty report. The reason for this is 
obviously due to the ease with which these mileages can be obtained. But this warranty 
population may be different "mileage wise" than the population as a whole. For example, 
appearance on the warranty report would be influenced by high mileage drivers. Thus, 
the basis for mileage calculation is very likely bias. 

If this method was to be employed, work would still have to be done to obtain an 
ultimate reliability function. Obviously, this method has potential for reliability 
estimation. 

A warranty reporting system provides data on all reported failures. It is usually 
expected that any reliability analysis be done from this same information source. How- 
ever, any failure reporting system that attempts to report all failures is usually 
lacking in the quality (or accuracy) of information. For reliability analysis a better 
strategy is to closely monitor a small subset of the population such that highly accu- 
rate data is obtained. In order to accomplish this, a network of select dealers have 
been identified for the specific purpose of product reliability analysis. The vehicles 
sold by these dealers are tracked, by failure reporting at the dealer, for a period of 
three years. The dealers were selected partially on the basis of high return rate of 
customers [2] . Thus, although the remainder of this report speaks of total sales, in 
actual application a subset of the total population is utilized, and total dealer sales 
for this subset is appropriate. 

In a typical manufacturing situation sales are going on continuously, and as units 
are sold they see a variation of use that depends on customer driving habits. Thus, 
one must consider this dynamic sales and use environment in order to estimate the mile- 
age on all vehicles in the field. The failure of a vehicle is related to accumulated 
mileage; however, mileage is only known for failed vehicles. The age of vehicles 
"time wise" is known, and this is indirectly related to miles of usage. 


* 
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Other compounding factors in this situation are the extremely low failure rates, 
and the lag in time from sales, to mileage accumulation, to failure reporting. For 
example, one percent failures on a subsystem during warranty would begConsidered a near 
panic situation. This indirectly implies a failure rate of A=0.8xl0~ /mi (assuming an 
exponential distribution). For a particular vehicle line the total model year produc- 
tion might be 600,000 vehicles. However, this production occurs over a 10 month period. 

Thus, at the end of the second month of production there may be about 10,000 vehicles 
actually in the customers hands. If we assume that all vehicles had experienced exact- 
ly 1,000 miles of use (probably an over estimate) then we would not expect to see even 
one failure although the failure rate is at a level we would like to detect. 

At this point the scope of the problem should be recognized. The next section 
considers two further approaches. These approaches are based on two distribution 
models that are widely used in this industry. 

PREDICTION MODEL DEVELOPMENT 

In order to estimate vehicle reliability, one approach is to select a failure dis- 
tribution model. The exponential distribution is widely accepted and is usually appro- 
priate for system level failures where a constant failure rate may apply. However, 
mechanical component failures are usually due to fatigue, corrosion, etc., and for 
these failures the Weibull distribution is a more appropriate and widely acceptable 
model. If the failure distribution is assumed to be exponential, the mean miles be- 
tween failure (MMBF) parameter can be estimated by determining the total accumulated 
mileage of the entire vehicle fleet and the the number of failures. Tf the failure 
distribution is Weibull, the parameters might be estimated by some variation of the 
method of least squares by estimating the number of vehicles with mileages that are in 
between the given failure mileages. A prediction method will now be discussed for each 
of the failure distributions. 

The model development will be confined to one vehicle line. It is assumed that 
the manufacturer knows the date the vehicles are sold and the mileage and date of a 
failure. This represents the usual information available for reliability estimation. 

Only one type or class of failure is considered, and it is assumed that every failure 
is returned to a dealer (i.e. reported). This assumption would generally be true where 
a warranty program is in effect and for major component repair specifically for the 
subset of dealers used in this study. 

As in any model development situation, several assumptions were made: 

1) The incidence of multiple failure on the same vehicle for the same compo- 
nent is assumed negligible during the period of time under consideration. 

Thus, all mileages to failure are assumed to be to first failure 

This assumption, unfortunately, is necessary due to 'the reporting systems in common 
use. Normally, it is extremely difficult for an engineer to obtain information on in- 
cidences of multiple failure because of the information-handling capabilities of the 
reporting system. This assumption has different implications depending on the specific 
subsystem under consideration. 

2) The distribution for vehicle miles per month is assumed to be normal. 

3) The mean of the distribution for vehicle miles per month is dependent upon 
the month of the year (seasonal effect) . 

4) The standard deviation of the distribution for vehicle miles per month is 
the same for all months. 

5) The number of vehicles leaving the population due to accidents, etc. is 
assumed negligible. 

6) Vehicles are sold uniformly throughout the month; therefore, in determining 
mileage in the first month of usage, the vehicles are assumed sold on the 
fifteenth day of the month. 

Exponential Failure Model 

For the exponential failure distribution the MMBF can be estimated by[l], 

MMBF*= T/r 

where T=total accumulated mileage for the entire vehicle fleet and r=total number of 
failures. The total accumulated mileage would be the summation of the mileages 
individual vehicles had accumulated from the date of sale up to the date at which the 
MMBF is estimated. This accumulated mileage is not known, as in the usual case, and 
hence, must be estimated. 
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The approach used to estimate the accumulated mileage (T) , as heeded in order to 
estimate the MMBF for the exponential distribution, will now be considered. First, two 
random variables will be defined. Let 

y k = total accumulated mileage of the fleet at the end of the k month 


and 


mileage of an individual vehicle in the month 


The accumulated mileage of an individual vehicle would be the summation of the mileages 
driven in each month from the month the vehicle was sold up to the end of the kth month. 
Since the vehicle was assumed sold in the middle of the month, the mileage in the first 
month of usage is taken as one-half of the mileage for the month. 


The accumulated mileage for all of the vehicles sold in the j month is obtained 
by multiplying the number of vehicles sold (S.) by the mileage random variable for the 

jth month ( x j)* The total mileage is the addition of accumulated mileage for vehicles 
sold in each month as follows: 

T ~ Y k ~ S 1 (vr- + x 2 + x 3 + . . . + x R ) + S 2 (~ + x 3 + x 4 + . . . + x fc ) + . . 


S x k-l S ^ 

k-1 + x k ) + b k ( 2 % 


or , 


% I S i x i + 
j=l 2 2 


k-1 k 

l [s, ( l 


x,)] 


i=l 


j=i+l 


The above equation holds while the vehicle model is still being sold. When the vehicle 
is out of model year, the size of the vehicle fleet is fixed. The equation has to be 
modified slightly as follows: 


hi 

j=l 


m-1 

l 

i=l 


k 

l 

j=i+l 


x.) + ( l S.)( 
2 i=l 


k 

l 

j=m-H 


x.) 


for k _> m, where m = last month of sales. 


Since the mileage on a vehicle is assumed 
accumulated mileage (y k ) is a random variable. 

MMBF* = i.i /r 


to be a random variable, the total 
The estimator for the MMBF is taken as 


where y is the mean for y k » To determine y^ consider the following. 

Under our assumption, we have E(x.) = y. for the j 1 -* 1 month, and the variance is 
2 J J 

o for any month. Then for k _> m, the average accumulate mileage is 



An approach for calculating a confidence interval for the MMBF will now be pro- 
posed. Epstein [1] has shown that where failures are accumulated over. an interval of 
mileage the confidence interval is 



(2T )/*a/2,2r + 2i MMBF i (2T)/x l-a/2,2r 

In our situation since, in many instances, r will be large, the Chi-square values can 
be approximated, and the confidence interval can be written as 

(4T)/(Z ft/? + /4r+3) 2 £ MMBF <_ (4T)/(v / 4?4 : 3 - Z^) 2 

where is the 100(l-a/2) percentile for the standard normal distribution. 


The confidence interval would be correct if T was exactly known; however, in our 
situation T is not precisely known but is assumed to be a random variable, and this 
introduces a further source of variability. Consider the following approach. One can 
obtain a (1-B) probability interval given by ^ and T^ Qwer that contains the random 

variable T = y^, with a (1-3) probability, since T is a sum of normally distributed 
random variables. So, the confidence limits on the MMBF are taken as 


(4T ) / (Z /0 

lower a/2 


MMBF < (4T )/(/4r+3 

— upper 


W 


This is considered to be a (l-a)(l-B) confidence interval for the MMBF. 


The above approach estimates a subsystem MMBF with a related measure of precision 
as expressed by the confidence interval. The approach requires knowledge of driver 
habits. A good study of driver habits might allow one to take a stratified sampling 
approach in order to decrease the variability introduced by driver mileage. 

The Weibull Distribution 

The Weibull failure model has a cumulative distribution given by 
F(t;0,B) = 1 - exp[-(t/9) 3 ], t > 0 

where 3 is the shape parameter or the Weibull slope, 0 is the scale parameter or the 
characteristic life, and the minimum life is zero. Taking the natural logarithm twice 
and rearranging the equation into the standard form for the dependent and independent 
variables gives 

ln(t) = -jjrln ( In + ln(0) 

which is clearly of the form y = (l/3)x + A, and will plot as a straight line on rectan- 
gular (x,y) graph paper. Although the following explanation is given as if it was 
based on plotting on Weibull paper, the thought is to use a least squares approach in 
a computerized procedure. 

Failure mileages establish the abscissa values for plotting on Weibull paper. 

The corresponding values of the cumulative distribution function F(t) are needed to 
establish the ordinate values, or one must know the fraction of the population failing 
prior to each observed sample value (i.e., failure). Several approaches are available 
for estimating F(t) [5]. However, it should be recognized that for our particular 
application the sample size (n) will be large, since production is high volume. Thus, 
in our particular case, we feel justified in simply using j/n where j is the ordering 
of the failure time. 


In the case of warranty data, many vehicles have not failed, and these must be 
treated as suspended items. The suspended items can be accounted for by giving the 
failure times an average order number [3,4]. That is, if the data contains only fail- 
ures, the items (failure times) are ordered and numbered (1,2, . . . ,n) from the small- 
est mileage to the largest mileage. With suspended items an average order number is 
assigned to the failure times considering all possible orders if the suspensions had 
continued to failure. In order to do this a new increment is determined [4]. This is 
reduced to an easy calculation procedure by the use of a counting increment (N.I.) 
which is 


j. _ (n+1) - (Previous Mean Order Number) 

1 + (Number of items beyond present suspended set) 

This increment is then added to the previous mean order number for each vehicle failure 
until another suspension is reached. 

The above method is applicable if one knows the number of suspended vehicles and 
their mileages. If vehicle warranty data is used, the mileages for the suspended 


m 

;*$• 

i 
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vehicles are not known, and must be estimated. A procedure of estimation for applica- 
tion of the Weibull distribution will now be explained. 


The proposed approach is to estimate the number of vehicles in the population that 
are suspended between the reported failure mileages. The first step is to estimate the 
mileage per vehicle based on the present varying age of the population. Let z, = the 

tVi 

random variable for miles per vehicle at the end of the k month. The miles per ve- 
hicle is estimated as the total mileage of the entire customer fleet divided by the 
total number of vehicles sold through the k^ 1 month, or 

k 

\ ' h 

Let J k 

hk - h 

where f ^ = the fraction of population sold in i*^ 1 month taken at the end of the k*"^ 1 

month. Therefore, for k < m 

k k-1 k 

z k = % I f ik x i + l [f ik ( ^ X i )] 
k j=l Jk 3 i-1 liC j=i+l 3 

Similarly, for k >_ m, and recognizing that f^ k = f^ are now constants, we have 
m m-1 m k 

z k * % l f , x + l [f.( l x )] + l x 

R j=l 1 J i=l 1 j-i+1 3 j=m+l 3 

The mean and standard deviation for the mileage per vehicle random variable would be 
as follows: For k < m. 


= ^ ^ f jk U j + E [f ik ( ^ M 3 } 

Z j=l Jk 3 i-1 lk j-i+1 3 


°l = %l f L ° 2 


k-1 _ 

l tq k (k-Do 

i-1 lk 


For k > m, 


% ^ f j y 3 
3=1 J 3 


+ l 1*1 t l ^)] + 1 u, 

i-1 j=i+l J j=m+l J 


i. T c 1 2 


l [f . (m-i)o ] + (k-m)a 
J 


The distribution of the mileage per vehicle, based on our assumption, would be 
normal with mean and standard deviation . The mileage to failure of the j * ^ ve- 
hicle is defined as t,, as reported. The fraction of the population below a particular 
reported failure mileage would be estimated as 

t .-p 

P(z > t ) = *I-J — -] 

J z 

where <1 ( • ) is the cumulative standard normal. The fraction of the population between 
two adjacent failure mileages (t. ^ and t.) is 


t i" y - 

P , *[-1— 
J z 




Thus, the number of vehicles from the entire customer fleet that have mileages 
between failures is 

k 

N. = ( l S.) P. 


Now, returning to the N.T. equation, the counting increment for the j 


failure mileage 


NT. = 
J 


(n+1) 

1 + h 

i=l 




)i 


where 


. th 


ON. = the i failure order number 
1 


.th 


NI. = the counting increment applied at the (j-1) failure to obtain the j 
^ order number 

The denominator is the number of vehicles beyond the present suspended data set, which 
equals the total number of vehicles sold (n) minus the number of vehicles with mileages 

below the j 1 ”* 1 failure ( £ N.). The failure order number for the failure is then 
i=l 1 


ON. 


ON. n + NI. 
1-1 J 


.th 


The estimate for F(t) at the j failure for this application is simply taken as 


R. = ON./n 
1 J 


With the rank values and the failure mileages, the Weibull failure distribution 
can be estimated at least two ways. First, the data can be plotted on Weibull graph 
paper, then a straight line can be drawn through the points producing an estimate of 
the population distribution from which the data originated. One could then make pre- 
dictions about the population. 

The second proposed method analytically estimates a straight line through the 
calculated data points by using weighted least squares. This method is explained in [7]. 

CONCLUSION 

The approaches presented in this paper are very elementary and many other consid- 
erations should be incorporated. For example, some customers will be lost at the end 
of the warranty period which terminates on mileage or age. An approach is needed for 
measuring and handling this consideration. Also, other basic approaches such as the 
use of nonpararaetric models [6] should be explored. 

Automotive reliability estimation presents unique problems, and it is hoped that 
others will be encouraged to continue research in this area. Ultimately, a good meas- 
ure of product reliability is an advantage to a company in that it indicates product 
weaknesses and also, over design. Through this knowledge a better product can be 
provided at a reduced cost. 
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INTRODUCTION 

In 1962 J. T. Duane of General Electric postulated that as long as a continued re- 
liability effort is maintained that cumulative failure rate and cumulative hours will 
plot as a straight line on log-log papet. His thesis was supported by five sets of 
data obtained during the developmental test phases. 

Since that time a number of papers have been published on various aspects of 
growth modeling utilizing the basic Duane model. Papers have been presented on the 
basic usefulness of the Duane Model (2, 3, 4, 7, 9, 12, 13), the relationship of the 
Duane Model to the Weibull failure rate function (7, 8, 12), methods of estimating 
parameters (1, 5, 6, 8, 10), and techniques for determining confidence limits for the 
parameters (1, 6, 8, 10, 11). 

The purpose of this paper is to present an application of the Duane Growth Model 
to equipment which has been design released and in production. A common element of 
papers cited above is that the data was obtained during development test programs. 

Sets of data have been obtained on two lots of different equipment types that are 
subject to production burn-in tests of 72 hours. Upon completion of the burn-in test, 
10 items from a lot were selected for a production reliability acceptance test per 
MIL-STD-781B test plan XVIII. 

A Duane Growth Model was fitted to the data and projected to the end of the re- 
liability acceptance test. This enabled management to determine whether the equipment 
had a reasonable chance of passing the reliability acceptance test. Both examples re- 
sulted in reasonable projections which were later compared to Duane Growth Models 
calculated for all test data through the reliability acceptance test. 

Duane Growth Model 


The following formulation for the Duane growth model has been used in this paper. 

m(t) = m t^ 
o 

where 

m Q = the intercept value of the MTBF (log m at log t = 0) 

$ = the growth slope 

m(t) = Cumulative MTBF at time t 

t = test time /equipment or equipment age 


The cumulative MTBF is based on the total test time and failures on all equipments 
of the burn-in test. 


m(t) = nt/F 


where 

n = number of equipments 
t = test length 

F = Total number of failures on all equipments 
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The major difference in this formulation compared to the applications of the 
growth model to development test data is in the definition of "t". For development 
test data "t" is the cumulative test time. However, for burn-in data it is more appro- 
priate to define "t" as the equipment age. 

This model can be used as a tool to predict the possible outcome of a production 
reliability acceptance test based on the acceptance criteria, number of units on test 
and equipment age at the start of the test. 

The parameters m and 3 were calculated using a weighted least squares technique. 
Each failure is weighted by its failure number, F.; each failure is plotted as though 
it occurred times at that point. L 

I F (E F log t log m (t)) - (E F log t ) (E F log m (t)) 

_ X X X 1 1 11 1 

E F E (F_^ (log t i ) 2 ) - [E F^ log tj 2 


log m Q = 


E log m^t) - 3 £ F^ log t. 


This method gives greater weight to the most recent data, which in reality is 
cumulative and contains all previous data. 


An estimate of the current MTBF (M(t)) of the equipment is obtainable from the 
Duane model by the following equation. 


EXAMPLE A 

This example consists of 59 items, each of which was subjected to 72 hours of 
burn-in. After completion of burn-in, 10 items were selected for a reliability 
acceptance test per MIL-STD-781B Test Plan XVIII. This test plan is a fixed test 
length of 9.4@ o , with an accept decision with 13 or less failures. 

The data of the burn-in test is given in Table I. The following results were ob- 
tained : 

Total Burn-in Test Time = (59) x (72) = 4248 

Number of Burn-In Failures = 18 

Observed Cumulative MTBF 236 

The required MTBF (0^) for the equipment was 800 hours. 

Total Reliability Acceptance Test Hours = 7520 

Projected Number of Failures 7520/236 = 31.9 

Since passing the acceptance test required 13 or less failures, this simplified 
analysis would make it appear that there was absolutely no possibility of passing the 
test. The purpose of a burn-in test is to eliminate early failure mechanisms and a 
decrease in failure rate is to be expected. The parameters of the Duane Growth Model 
for this set of data are given below and plotted on the figure for Example A. 

8 = .326 

M = 40.29 

o 

In order to project the expected, number of failures, we note that the ten systems 
will enter the reliability acceptance test with 72 hours of operation and end with an 
additional 752 hours for a total of 824 hours. To determine the number of failures 
per equipment, we calculate: 



K5& 


u 



m(72) = 162.4 hours + .44 cumulative failures 
m(824)= 359.6 hours + 2.29 cumulative failures 

The expected number of failures per equipment is 2.29 - .44 or 1.85 failures 
during the reliability acceptance test. For 10 equipments we expect 18.5 failures; 
considerably less than the 31.9 projected without the benefit of a growth analysis. If 
we assume Poisson, the probability of 13 or less failures is approximately 12%. 

After completion of the reliability acceptance test, during which 9 failures 
occurred and an accept decision made, a Duane Growth Model using both the burn-in data 
and the additional data obtained from the reliability acceptance test was applied. 

This data is presented in Table 2. The parameters of this Duane Growth Model are given 
below and plotted on the graph of F.xample A. 

8 = .351 

M = 38.1 

o 

The burn-in data appears to have under-estimated the growth rate resulting in a 
slight over-estimate of the expected number of failures. 


EXAMPLE B 

This example consists of 49 test items, each subjected to burn-in of 72 hours, 
with 10 items selected for a reliability acceptance test with 0 q = 1200 hours. 


The 

data of the burn-in test is given 

in 

Table 3. The following results 

tained : 

Burn-In Test Time = (49) x (72) 

= 

3528 hours 


Number of Burn-In Failures 

= 

14 Failures 


Observed Cumulative MTBF 

= 

252 Hours 


Total Acceptance Test Hours 

= 

11,280 Hours 


Projected Number of Failures 
11,280/252 

_ 

44.7 


The conclusion of this analysis is that passing a reliability acceptance test with 
13 or less failures would be highly improbable. 

Utilizing a Duane Growth Model similar to Example A, we obtain for this set of 

data 

8 = .483 

M = 23.6 

o 

This line is projected on the graph for Example B. The following is the pro- 
jected number of failures, where each item will have 1200 hours of operating time at 
the end of the acceptance test. 

m(72) = 186 hours + .39 cumulative failures 

m(1200) = 725 hours ^ 1.65 cumulative failures 

The expected number of failures per equipment during the reliability acceptance 
test is 1.65 - .39 or 1.26 failures or for 10 equipments we obtain 12.6 expected 
failures. If we assume a Poisson we would project a 60% chance of passing the reli- 
ability acceptance test. 

In this case 9 failures occurred during the reliability acceptance test, resulting 
in an accept decision. Table 4 gives the combined data for the burn-in data and the 
reliability acceptance test. The following Duane parameters were obtained and plotted 
on the graph of Example B. 

8 = .434 

M = 29.3 

o 


TABLE 1 

EXAMPLE A - COMPARISON OF OBSERVED AND DUANE 
MODEL CUMULATIVE MTBFS OBTAINED DURING BURN-IN TEST 


EQUIPMENT 

CUMULATIVE 


CUMULATIVE 

OBSERVED 


DUANE 

DUANE 

AGE -HOURS 

EQUIPMENT 


EQUIPMENT 

CUMULATIVE 


CUMULATIVE 

CURRENT 


HOURS 


FAILURES 

MTBF 


MTBF 

MTBF 

8 

472 


6 

78.7 


79.4 

117.8 

16 

944 


8 

105.5 


99.5 

147.6 

24 

1416 


13 

108.9 


113.6 

168.5 

32 

1888 


16 

118.0 


124.7 

185.0 

40 

2360 


17 

138.8 


134.1 

199.0 

48 

2832 


18 

157.3 


142.4 

211.3 




TABLE 2 





EXAMPLE 

A 

- COMPARISON 

OF OBSERVED AND 

DUANE 



MODEL CUMULATIVE MTBFS OBTAINED DURING 

BURN-IN TEST 



AND EXTENDED THROUGH PRODUCTION ACCEPTANCE TEST 


EQUIPMENT 

CUMULATIVE 


CUMULATIVE 

OBSERVED 


DUANE 

DUANE 

AGE -HOURS 

EQUIPMENT 


EQUIPMENT 

CUMULATIVE 


CUMULATIVE 

CURRENT 


HOURS 


FAILURES 

MTBF 


MTBF 

MTBF 

8 

472 


6 

78.7 


79.1 

121.8 

16 

944 


9 

105.5 


100.8 

155.3 

24 

1416 


13 

108.9 


116.2 

179.0 

32 

1888 


16 

118.0 


128.6 

198.1 

40 

2360 


17 

138.8 


139.0 

214.2 

48 

2832 


18 

157.3 


148.2 

228.4 

80 

4139 


20 

206.9 


177.3 

273.2 

88 

4227 


21 

201.3 


183 . 4 

282.6 

96 

4323 


22 

196.5 


189.1 

291.4 

104 

4414 


23 

191.9 


194.5 

300.0 

160 

4974 


24 

207.2 


226.2 

348.5 

400 

7374 


25 

294.9 


312.0 

480.7 

584 

9201 


26 

353.9 


356.3 

549.0 

800 

10801 


27 

400.0 


397.9 

613.1 




TABLE 

; 3 





EXAMPLE 

B 

- COMPARISON 

OF OBSERVED AND 

DUANE 



MODEL CUMULATIVE MTBFS OBTAINED DURING 

BURN-IN TEST 


EQUIPMENT 

CUMULATIVE 


CUMULATIVE 

OBSERVED 


DUANE 

DUANE 

AGE -HOURS 

EQUIPMENT 


EQUIPMENT 

CUMULATIVE 


CUMULATIVE 

CURRENT 


HOURS 


FAILURES 

MTBF 


MTBF 

MTBF 

8 

392 


5 

78.4 


64.5 

124.8 

24 

1176 


12 

98.0 


109.6 

212.0 

48 

2352 


13 

180.9 


153.2 

296.3 

56 

2676 


14 

191.1 


165.3 

319.7 




TABLE 

; 4 





EXAMPLE 

B 

- COMPARISON 

OF OBSERVED AND 

DUANE 



MODEL CUMULATIVE MTBFS OBTAINED DURING 

BURN-T.N TEST 



AND EXTENDED THROUGH PRODUCTION ACCEPTANCE TEST 


EQUIPMENT 

CUMULATIVE 


CUMULATIVE 

OBSERVED 


DUANE 

DUANE 

AGE -HOURS 

EQUIPMENT 


EQUIPMENT 

CUMULATIVE 


CUMULATIVE 

CURRENT 


HOURS 


FAILURES 

MTBF 


MTBF 

MTBF 

8 

392 


5 

78.4 


72.3 

127.7 

24 

1176 


12 

98.0 


116.8 

206.0 

48 

2352 


13 

180.9 


157.3 

277.9 

56 

2676 


14 

191.1 


168.2 

297.2 

80 

3449 


15 

229.9 


196.3 

346.8 

128 

5181 


16 

323.8 


240.8 

425.4 

168 

5581 


17 

328.3 


270.9 

478.6 

248 

6321 


18 

351.2 


320.8 

566.8 

264 

6481 


19 

341.1 


329.6 

582.3 

536 

9201 


20 

460.1 


448.2 

791.8 

544 

9281 


21 

442.0 


451.1 

797.0 

816 

9694 


22 

440. 6 


537.9 

950.3 

1116 

12627 


23 

549.0 


616.2 

1088.7 


CONCLUSION 


Both examples have shown that the Duane Growth Model applied to burn-in data will 
obtain reasonable results when used to project the probability of passing reliability 
acceptance tests. The assumption of continued growth during the reliability accep- 
tance test seems to be justified in the two examples presented. 
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A FMECA'S USE 


BY WHOM AND HOW 


Richard R. Erick, Supervisor of Reliability 
Quality Assurance Dept., Bendix Energy Controls Div. 
South Bend, Indiana, 46620 


INTRODUCTION 

We all know what is required in a FMECA, a Failure Mode, Effects and 
Criticality Analysis. We have seen many types of formats. Most of them 
have headings similar to the two examples shown in Table 1. 


Table 1. FMECA Format Headings 


Example "A" 

Identification Name/Number 
System Name/Number 
Part or Assembly Number 
Cause 

Failure Mode 
Effect of Failure 
Hazard Classification 
Remarks 


Example "B" 

System Model 
System Name/Number 
Component Name/Number 
Reported By & Date 
Revised By & Date 
Item Number 
Function 
Failure Mode 
Primary Effect 
Design Philosophy 
Method of Detection 
Hazard Classification 
Failure Rate per Mode 
Failure Rate Summary 
Approvals 


The fact that we have been required by contracts and specifications to 
provide detailed studies of each part is very much old hat to most of us. 


The use of a FMECA, in my opinion, 
has changed. FMECA ’ s have gone from: 
"Fictitious Material Easily Cataloged 
Ashcan" or maybe "For Management, Extra 
Costs Accumulated" to "Factual Mater- 
ial Easily Compared to Actuality". This 
change in the importance of the FMECA 
has been brought about, at least in the 
military aircraft accessory industry, 
again my opinion, by the ever increas- 
ing emphasis for the "big R - easy M" — 
maximum reliability with a minimum of 
easy maintenance. And, I believe that 
the increased emphasis placed lately on 
product liability in the commercial in- 
dustry has placed more rather than less 
importance on the use of the FMECA. 



It is the purpose of this paper to 
describe some practical , day to day ways 

in which the FMECA 1 s use can be upgraded to meet this increase in need. 
At least, to present ways which have helped me. These ways involve who 
produces the FMECA 1 s , who uses the FMECA’ s and how they are used. 

©1978 American Society for Quality Control - 



CUSTOMERS USE FMECA ' S 


It is a known fact that customers want a "warm feeling" about our 
product's ability to operate satisfactorily long enough to meet contract 
requirements — not just long enough to get out of the warranty period — 
but through years of service life as well. Words like "design based on 
past experience" or "designed by a 
leader in the industry for the past 25 
years" haven't cut the mustard in 
FMECA 's like they might have in the 
past years. Management teams have 
been demanding design PROOF that the 
products have actually been designed, 
not just put together. It is a known 
fact that customer "A" is well aware 
of the major problems you have been 
having with customer "B". And, cust- 
omer "A" expects proof that the design 
for him has actually addressed and has 
corrected past problems like those of 
customer "B" with effective changes 
and corrective actions. 

FMECA reading is dry work. Work 
that few people have had-or have taken 
the time necessary to accomplish the 
task adequately. But, I am convinced that the customers have read them. 
Especially when a problem has happened. Among other things, they want 
the answers to questions like: "Was it considered in the FMECA?", "How 
did you design it this time to prevent that type of problem?". These 
questions come almost on top of the usual ones like: "What are you doing 
about it?", "When will it be fixed?", and, "How many more like that one 
have you already slipped past me?". 

So, maybe I have convinced you that customers want from this report 
the warm feeling that our companies designed the device. Designed them 
from design policies and have it all documented. And, they expect the 
assurance that the documentation is easily retrievable, not buried away 
in three or four designer's notebooks, scratch pads, or in a waste basket 
somewhere. I have found that the FMECA is that documentation to provide 
this type of assurance. But, does anybody else use it? They sure do, — 
that's my answer. Or, at least, they sure should. And, I maintain, they 
sure can if the FMECA is done practically. 

DESIGN ENGINEERS USE FMECA ' S 

Since design engineers have been given copies of the FMECA' s -- and 
have read them, some definite improve- 
ments have been made in the designs. 

Improvements that were made at a time 
in the program when all it took was an 
eraser, — and maybe 10 hours plus about 
a week's slip in the delivery schedule 
— but it is the cheapest point in the 
ball game to catch an oversight. When 
the FMECA indicated that the part "may 
be assembled backwards", sometimes a 
light came on in the designer's mind 
of a way in which he was able to alter 
the design and prevented it from being 
assembled backwards. It might just be 
that the designer was actually con- 
vinced that whoever it was that wrote 
down "Corrosion" as a possible cause 
for a failure was a person who had 
actually experienced such a cause for 
a failure on one of the past designs 
to the point that it was of major con- 
cern to him. And more attention was 







actually paid to the manner in which he was really trying to prevent 
corrosion on this new design. Therefore, maybe a better way was used. 

After seeing "external fuel leakage" as an effect for one "0" ring 
after another, around the outside of his beautiful design, the designer, 
the sole owner of the eraser, sometimes actually found a way to eliminate 
a few of these kinds of parts-- or the parts that required the "0" rings 
for that matter. I know, designers are like most of the rest of us. They 
do not like to have their work questioned. But, when they realize that 
the FMECA is an honest attempt to help them to prevent past problems from 
turning up once again on new designs, they have looked upon this tool of 
management in a somewhat different light. 

INSPECTION ANALYSTS USE FMECA ' S 

Many times in the past, when I wrote FMECA' s, I knowingly indicated 
— in the area of the FMECA sheet explaining why this part will never 
fail that way -- that "Inspection will check it. Quality will catch it, 
pressure test it, verify that the operation has been done properly and 
checked". And, chills went up and down my spine. My gosh, will they -- 
really? How can I be sure that some- 
body even knows (-or cares-) that I am 
telling our customer this. It was soon 
clear that we had better get a copy of 
the FMECA for the Inspection Analysis 
Manager, also. But, then maybe - way 
back then - I had conversations like: 

"What's this for?" 

"You expect me to read that 'Manhatten 
phone book' of a thing?" , or, 

"Oh, yea, thanks. I'll get around to 
reading it when I can find the time." 

Maybe you have had some conversations 
like that, too. 

They previously didn't know what 
a FMECA was. They didn't know what was 
in it that will be of benefit to them. 

They didn't know how to use it. Maybe 
one or more good ideas from the FMECA 
might pay back a whole lot. It was agreed that they should be told (and 
convinced) that it was to their benefit as well as the company's benefit 
for them to actually USE it. There are ways to help them to use it, but 
before going into that group of suggestions concerning ways we have found 
to organize a FMECA so that they actually help the users, permit me to 
discuss two more major users (and uses) of the FMECA. 

ASSEMBLY PROCESSING'S USE OF FMECA ' S 

A group of people at our plant write out detailed instructions as to 
how each part is to be assembled and where. Great big thick books-~for 
large fuel controls that are very complicated- (in that they have lots of 
parts, I mean). Full of pictures, sketches, arrows, instructions, and 
the like. It was realized that this group should know that if this tiny 
cotter pin wasn't properly assembled and spread that it could bind up on 
the adjacent part. These are the people who make up the check sheets or 
something like that to make sure that all of the taper pins have been put 
in and that they are all properly spread because if one of them falls out 
all sorts of problems may start to happen. 

Oh, sure, this is very routine stuff. Even without the FMECA. Not a 
thing new about that. But, it was found that this group was more concern- 
ed about these special instructions and cares after they had some way of 
knowing the major consequences of these routine operations if they might 
accidently be done improperly. Again, they became convinced that these 



undesirable events have most likely happened before — at what expense 
God only knows — and they were more likely to heed the warnings of the 
FMECA, and heed them more firmly than before they knew that the FMECA 
was in existence or before they used it. 

RELIABILITY ENGINEERING USES FMECA* S 

"Naturally", you say. "Didn't they write the thing in the first 
place?" Yes, no doubt they did. But, I maintain that this task only 
started the ball rolling. Now, they should use it as well. 

It has been found very productive to use it to track actual failure 
reports back to the individual piece parts, the item(s) in the design 
that caused the problem in the first 
place. Also use it to upgrade failure 
mode lists, to find out actual failure 
rates for the same kinds of parts used 
in different applications. Benefits 
are possible by proper organization of 
the FMECA so that it can be used. 

Published failure rates by various 
standards, handbooks, etc., seem to be 
loosing favor with the military and 
customers of the military. They are 
no longer accepted at face value as a 
good source of failure rates. Actual 
experience sometimes is too far from 
the anticipated failure rate. Designs 
estimated from such "Cookbook" values 
don't seem to come up to expectations 
concerning the failure rates like they 
should have. It seems that what is 
wanted is YOUR ACTUAL experience with 
these parts, YOUR records, YOUR failure rates. That seems to be what 
really counts. 

This is great if you have the data already. Maybe electronic de- 
vice manufacturers have such data. I don't believe that very many of 
the hydromechanical device manufacturers are very far along this road 
yet when it is the specific part type, part application — as you use it 
— that is what is desired. I have found that the FMECA is one way of 
logging or cataloging failures so that factual failure rates can be pro- 
vided. But, it is murder to go back to the file drawers and try to pull 
that kind of stuff out of the past field service reports, quality inves- 
tigation results, wires, etc. Yet, it has to start sometime. No better 
time than your next FMECA, you next new contract. Having a copy of the 
FMECA organized by part number, another copy organized by part type, and 
still another copy by functional groups as well as the copy set up per 
the Bill of Material setup will be found to be most useful, I am sure. 
With these, it is much easier to catalog actual failures on the back of 
the appropriate FMECA sheets, with the dates, and the other reference 
data you may need to get back to the actual facts. Now, with this kind 
of data organization, you can get factual failure rate data at any point 
in the program. 

After all, since the FMECA can be organized by part number and also 
by part type, all we need to know to play the failure rate game is the 
amount of test time in the time period being studied. If every hour of 
time means 50 "0" rings get tested, 20 springs, 5 bellows, 15 brazed 
joints, etc., then we can begin to get an idea of part type failure rate 
from our own data. If we know that it was a failure of a brazed joint 
that caused the malfunction, how about that -- we are even getting pro- 
cess failure rates. Now, we are getting somewhere. With data like that 
the statisticians can have a field day with point estimates, 90% confid- 
ence intervals, growth curves, and the like. With that kind of data we 
can develop our own cookbook of failure rates. And, isn't that just 
exactly what our customers have been wanting us to do all along? I am of 
the opinion that the FMECA organization is a very good place to start. 




WHO DOES WHAT 


The message of this paper was drastically altered by a question I 
was recently asked after my abstract was prepared and submitted for the 
review. The question was "Who uses the FMECA if the Design Engineer does 
not?" I hope that the discussions just completed have helped to answer 
that question. I hope that my ideas might be found to be helpful to you. 
But, another part of my answer concerned who writes FMECA' s. My answer 
to the above question included the statement that Design Engineers should 
WRITE a major portion of the FMECA as well as use it, with Quality Assur- 
ance doing the rest. 

So, if you are still with me, let me take a few minutes to pass on to 
you my suggestions on the who, what, and how of FMECA writing and organi- 
zation, as I have seen it from our needs and our own point of view. 

ORGANIZE FMECA' S FOR USE 

The FMECA has to be organized in 
such a manner that it is a ready tool 
to assist in answering such questions 
as "How many external plugs & "0"rings 
in this thing could cause an external 
leakage of fuel due to being damaged 
at assembly?" "How many external plugs 
are pressure tested to 100-150% of the 
maximum operating pressure in the hole 
closed by this kind of a device?" The 
answers to these questions are expect- 
ed from the Quality Assurance and Rel- 
iability Engineer many moons after the 
design calculations have been made. 

The FMECA is the major candidate as to 
where to document the answers to these 
questions so that the answers can come 
within a few hours, not days or weeks, 
or never. 

These answers, and others like 
them, have been found much easier to 
get after the FMECA has been presented 
by part number, one part number to a 
page. Copies of the FMECA can then 
be arranged, as stated earlier, by the 
part number, type, functional group or 
any other way desired. When several 
items are discussed on one page, or a subassembly is talked about in only 
general terms, it becomes much more difficult to hunt for the answers. 

WHO STARTS -- WHO FINISHES 

What do I mean by "start". Start the FMECA by identifying each part 
on the sketch, schematic diagram, or layout. Castings, threaded inserts, 
soldered joints, cotter pins, sliding fits, etc., each and all must be 
identified. This may be much more difficult to do from a schematic dia- 
gram, but if it is on the schematic diagram, it no doubt is 'critical or 
at least important to the operation of the device. I know you have heard 
it at lease twice before, but use one FMECA sheet per part number. Start 
by filling in the part name, identification number or something so that 
you will be able to track the FMECA sheet back to the schematic diagram. 
Put in the most likely failure modes and the possible causes. Maybe even 
take a stab at the primary effect. "Most likely", to me, means three (3). 
Three types of problems that have actually occurred and have occurred 
most frequently in the past on similar products with the same types of 
parts. If only one such problem exists, or even no problems at all, -so 
much the better. Say so on the FMECA sheet. Let it be known that, in 
your opinion, no problems have been reported based on actual experience- 
yours or your company's. But, forget about the fifteen (15) or so fail- 
ure modes that you could think up that might just possibly happen. 




OK, start has been defined. So, who starts? From my point of view, 
the Quality Assurance Engineer starts. After all, if the Quality Engi- 
neer doesn't know what the past problems have been in the inspection, 
assembly, test, customer's plant and field service areas, at least he 
will know who to ask. Then, feed these sheets to the Design Engineer. 

At least, that is our way. And, one plea, request, mandate, or order 
to: "Give me facts, man." 

"What facts?" 

"Design for reliability facts." 

"What does that mean. All of my designs are reliable. 

I never designed anything to bust." 

"No, but sometimes they did. How come?" 

"Well, nobody told me about them. I can't be expected 
to be a mind reader. Why didn't somebody tell me 
about them, sooner." 

How sweet it is to be able to now go to the design engineers with 
the FMECA sheets and NOT get that kind of a response. You see, the de- 
sign engineer must be convinced that he should "do his thing" with his 
design calculations, etc. , and "that thing" is to put them into the FMECA 
sheets so that his data becomes a part, a very major part, of the over- 
all picture that is being painted by the FMECA to give that customer the 
"warm feeling" he needs to decide on your company as a winner. 

DOCUMENT ALLOWABLES 

Not many of us, I don't believe, have gotten to the full use of the 
theoretical approach of designing for reliability, where the "nines" come 
out of the calculations like popcorn. But, we must know from the drawings 
what the actual stress is going to be in a part, like a spring. This is 
part of the picture. What the design engineer knows, in addition to this 
data, is that this stress is actually only 86% (or some such number) of 
the allowable stress usually used for this application by your company. 
But, unless he (the Design Engineer) tells somebody that, there is hardly 
any way to really be comfortable that it actually won't fail due to the 
stress that he said was going to be in the part. We need to tell the 
customer the allowables, as well as the actuals. Tell him in the FMECA. 
Stress, deflection, load, volume fill, percent squeeze, torque, you name 
it. Factors of safety, margins of design, fatigue ratios, are the kinds 
of data needed from the design engineer when he fills out the rest of the 
FMECA sheet. 

I am sure that you are familiar with some of the text books that 
have been written on the subject of designing for reliability. Actual 
design criteras have been developed by probably every company designing 
products. Some of these policies or criteras are public, some are your 
private property. I don't think that it is necessary to give away the 
barn in the FMECA, but, I do believe that it is necessary to provide the 
assurances that the designs have been checked against your standards and 
acceptable to them. These design policies have been needed for review, 
at least, by customers, at various times during a contract. 

WHEN A FMECA IS USED 

The last of the who, what, how, why and when is when. When to use a 
FMECA. At design reviews, naturally. 

But, we have found that it can be used 
at many more places, at other times 
than just there. I have attempted to 
tell you about a few of them by means 
of this paper. Benefits have been made 
from FMECA' s. It is my opinion that 
more benefits can be obtained from 
them. 

It is hoped that the ideas presented in this paper, basic as they 
may be, have at least started you thinking of ways you and your company 
can get more use from FMECA' s, by whom, and how. 
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TESTING FOR PRODUCT LIABILITY 


Steve Kozich, President 
Quality Audit Company 
Whittier, California 
(213) 696-1423 


HISTORY - WE CAN DO IT 

Product liability is new, however, product testing has been with 
us for a long time. The most sophisticated testing and evaluation 
started in the aircraft industry which included test pilot evaluation 
with some pre-flight safety testing. As the aerospace industry became 
more sophisticated and some systems no longer had test pilots, the 
Pre-Flight Rating Test (PFRT) began to include expected environments. 
Our early missiles were designed to a model specification, tested to 
an acceptance test, qualified for environments to a Qual Test Speci- 
fication and then safety considerations at various environments were 
investigated during the Pre-Flight Rating Test. With all of this our 
early missiles failed and the requirement for reliability was identi- 
fied. Then after the pilot-tested DC-3, reliability was transferred 
to Minuteman calculate reliability technology. The aerospace industry 
proved that good drawings and change control were necessary to assure 
what was tested matched what was furnished. In order to increase the 
price of aerospace equipment, the aerospace industry convinced the 
Department of Defense that good drawings and control was a new science 
called Configuration Management. In commercial products the computer 
industry has done the best job of assuring performance partly by design 
and secondly by specifying the environment of use. There is little 
doubt that both a reliable and safe product can be furnished to the 
consumer. The industrial technology and techniques of evaluation are 
available to industry. 


- INDUSTRIAL BEHAVIOR - 

Product liability laws have changed the pressure points and 
industry is now trying to get organized to deal with the problem. 

Many consumer demands are unreal, however, many industries are 
still working without good drawings and operational knowledge of what 
they produce . 

The most advanced industry may well be the toy manufacturers. The 
pressure is currently being placed on bicycles and skateboard manu- 
facturers. And the most stubborn is the automotive industry who has 
not responded since 1950, when we first saw smog, without daily fights 
and pressure delays in dealing with Congress. It is difficult to fault 
the automotive industry in design control, quality control or the 
product they provide. However, the automotive industry has created 
considerable problems with smog, and if they were not so big and 
wealthy, they would have to solve the automotive exhaust problem. 

On the other hand, the manufacture of football helmets is treated 
unfairly. No claims come from the professional ranks where trainers 
maintain the equipment and fit the proper size. Most claims come from 
the high schools who think the reason for helmets is to identify the 
team's color and each player should have one. 

- SOME INTERESTING PROBLEMS - 

Products manufactured should have some statute of limitation for 
products sold. Being held to the best known technology of the current 
day as the standard for products manufactured in the past is unrea- 
sonable. The National Product Liability Council, working on product 
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liability legislation, identified four major difficulties: 

1. First, there is no statistical proof available to prove in 
terms satisfactory to a crisis oriented Congress, that there 
is, in fact, a crisis. The reason, to anyone who has been 
involved as a buyer of large volume of insurance, is very 
simple. In product liability claims, in addition to the fact 
that a product may have been in existence many, many years 
before an injury occurs, it is normally eight to ten years 
before ninety percent of claims occurring within a year are 
paid. In other words, it will be 1985 or 1987 before the 
1977 claims are ninety percent paid. Insurance companies and 
insurance rating organizations are just not geared to make 
predictions of this nature for many reasons which are not the 
subject of this paper today. 

2. Second, there are strong entrenched special interest groups, 
which at every opportunity, cloud the issues. One such group, 
attorneys engaged in the injury business, blame the insurance 
industry. Another such group, the insurance industry, blames 
the manufacturer, pointing out that a manufacturer should have 
a strong safety program and that there are some products that 
underwriters today just don't believe deserve, in their mind, 
to be manufactured. 

3. The third hurdle, in our opinion, is that industry and busi- 
ness in general, has a bad public image. As consumers, all 
of us in industry are enjoying a life which is better than 
that of a millionaire of a mere one hundred years ago. Yet, 
it appears that in our capacities as consumers, we give little 
credit to the success of American industry in making all of 
this possible. 

4. The fourth difficulty is that any diversity of opinion that 
industry may have among itself will only be used by the 
special interest groups to stall and defeat a bill. We need, 
if at all possible, a unified approach that we can believe 

in as consumers, and not one that sounds like a special 
interest approach. 


- REDUCING THE RISK - 

Product liability has had several major impacts on industry. 
Leisure time products were high on corporate acquisition lists. Now, 
due to many consumer claims, many corporations are selling off leisure 
groups. The development of new products is being restricted if there 
is any sign of product liability exposure. Most corporations involved 
in mergers and acquisitions check very closely on past product liabi- 
lity claims and increases in insurance renewal rates. The reaction to 
products with high product liability risk is to quietly move them 
aside. This will develop two basic problems for the consumer-higher 
cost and difficulty in obtaining such products. 

Insurance companies will take more interest in the evaluation of 
manufacturers. They will look for quality c ontrol programs which mini- 
mize the chance of product failure. It is expected that design reviews 
will be completed to assure consideration for product safety and 
reliability . 


- LOOKING AT NEW METHODS - 

The transfer of technology in materials, processes, design con- 
cepts and product evaluation has generally followed from the aerospace 
industry to other industries. In product liability, other industries 
can now look to the toy industry for leadership. These designs have 
always been high on misuse, no maintenance and frequent drops. It 
has been ten years since the lady called the quality control manager 
of a leading toy company to ask for a new set of instructions because 
her daughter ate them. As the quality control manager laughed and 


offered a new set of instructions, she completely wiped the smile off 
his face with a simple question: "Do you know if the ink or paper 
could be poisonous?" This was a true surprise and concern that to most 
quality control managers had nothing to do with the product. 

The toy industry has developed product qualification practices by 
watching children play with toys through one way mirrors. Close 
attention is given to noise, temperature, color, size, weight and, 
above all, product safety. Sharp edges and points have been eliminated 
and the total product is qualififed to include unexpected ideas of the 
user . 


- NEW PRODUCT DEVELOPMENT REQUIREMENT - 

Each phase of the product life cycle (i.e. research, development, 
design, production, operation and disposal) requires specific tasks to 
be completed. In research each task is investigated on its own merit 
and a bank of knowledge is collected and catalogued for possible use. 
The development phase basically identifies a need and matches methods 
and combinations to develop a product concept to serve a specific need. 
In the design phase, product is completely identified and tested to 
assure proper operation. Testing is then extended to include proper 
operation in each environment (i.e. life, temperature, salt spray, 
humidity, vibration, shock, sand, dust, rains, sunshine, etc.) in which 
the product is expected to operate. The design phase is completed when 
the product has been completely identified and qualified by test for 
each environment. The production phase requires a complete check of 
the drawings and inspection of the hardware to assure complete compati- 
bility. Any changes in the design should be completely requalified and 
the product definition should be continuous so that each configuration 
can always be included in the active drawings and technical data 
package. The definition and hardware must always match with all non- 
conformances controlled. As the product enters the operational phase 
the manufacturer is required to provide spare parts and assure that 
they are available. The user has to be trained for proper operation. 
Both the manufacturer and user must work together in developing a 
maintenance plan and operational system to assure proper equipment 
utilization. Nuclear systems have now made us more aware of the 
disposal phase. We cannot store nuclear waste in junk yards like old 
automobiles . The above sequence is followed for appliances, aircraft, 
boats, farm equipment, consumer products, trains, ships and nuclear 
installations. The big problem is scheduling and completing each phase 
in the proper sequence. 

Product liability is adding a new requirement: safety consider- 
ations for failures and wearout. Wearout and failures can no longer 
cause side affects which are unsafe. The manufacturer is responsible 
to control conditions and environments during problem periods. The 
user must assure safety of removal, repair, installation and putting 
the equipment back in operation, however, if he has any problem, you 
can be sure the manufacture is responsible. 
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PROCUREMENT QUALITY ASSURANCE OF P C BOARDS 


Virgil T, . Bowers 
Procurement Quality Assurance 
MICRO SWITCH 
A Division of Honeywell 
Freeport, Illinois 61032 


First, I would like to position MICRO SWITCH for you and give you some idea of 
why quality printed circuit boards are so important to us. The precision snap action 
"Micro Switch" was invented in 1932 by an employee of the C. F. Burgess Battery 
Company of Freeport, Illinois. MICRO SWITCH began as a company as a spin-off from 
Burgess in 1937. The small precision snap action switch was a technological break- 
through that led to a lively industry. The company became a division of Honeywell in 
1950 and our catalogs today list over 30,000 different switches and industrial 
controls. We employ over 3,000 people in and around Freeport, and have additional 
plants in North Carolina and Massachusetts. We also have manufacturing facilities 
in Canada, Scotland, Germany and Japan. 

The. 1960's were the era of pushbuttons for switch manufacturers. Tn 1964 MICRO 
SWITCH introduced a mechanical code generator that we named the KB switch. In 1966 
the KB switch was modified by substituting the reed switch for mechanical contacts. 

In that same year MICRO SWITCH began building custom keyboards using reed capability. 

But customers kept asking for more wiring and more encoding to meet their hard- 
ware needs. Our high construction costs were discouraging. After extensive market 
research we decided to enter the market as a full keyboard supplier. We made that 
decision because we believed prices for electronic keyboards (then about $250) were 
too high, and because we had just experienced the second breakthrough in the 35 years 
of our history. This breakthrough was the conversion of the Hall effect (known since 
1879) into an integrated circuit chip complete with trigger circuitry and amplifica- 
tion. In other words, our engineers created a bounce-free electronic switch, compat- 
ible with nearly all forms of solid state logic, that would operate at a rate of 
100,000 times per second and offered nearly infinite life because actuation was by 
a magnet, which never even touched the chip. Price of this solid state keyboard? 
About $100. Price is still under $100. 

Today MICRO SWITCH is probably the largest non-captive maker of electronic 
keyboards in the world. Which makes us a large consumer of printed circuit boards, 
primarily for keyboards, but also for our photoelectric and proximity switch lines. 
Making sure that MICRO SWITCH keyboards were as reliable as possible led to the 
invention of some award-winning computerized test equipment, a one per cent AQL and 
the only two-year warranty in the keyboard business. It also led us to the sin of 
over-specing printed circuit board requirements. This is how we approached this 
problem. 

Our investigation started as a result of the excessive number of non-conforming 
lots of purchased printed circuit boards being received at MICRO SWITCH. The 
objectives were to reduce the overall percentage of defects and to increase the 
vendors' awareness of the problem areas. 
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REVIEW OF PRINTED CIRCUIT BOARD INSPECTION RESULTS JANUARY-NOVEMBER 


VENDOR 

LOTS 

RECEIVED 

LOTS 

NON-CONFORMING 

% 

NON-CONFORMING 

Vendor A 

294 

121 

41.2 

Vendor B 

397 

155 

39.0 

Vendor C 

48 

19 

39.6 

Vendor D 

225 

109 

48.4 

Vendor E 

128 

88 

68.8 

TOTAL 

1,092 

492 

45.1 


As a result of the high percent of non-conformance shown above, we needed to 
know the dollar effect of these non-conforming lots, so we ran an analysis of the 
five printed circuit board vendors vs. all vendors supplying purchased parts. The 
data was provided by our Monthly Vendor Analysis Report for the eleven-month period 


REPORT PERIOD ANALYSIS ( January-November ) 

1. Number of active vendors in the study 787 

2. Number of printed circuit board vendors supplying lots 5 

3. Total number of lots inspected 18,621 

4. Total number of lots rejected 1,507 

5. Total number of lots supplied by the five printed 

circuit board vendors in Point 2 above 1,092 

6. Total number of non-conforming lots supplied by 

the five vendors 492 

7. The cost to process 1,507 rejections at $40 average 

rejection cost $60,280 

8. The cost to process 492 rejections from five 

printed circuit board vendors $19,680 


REPORT PERIOD CONCLUSIONS 

1. .6% of our total vendors are responsible for 33% of our total non-conforming 
lots and total cost to process, while supplying only 6% of our lots. 

2. The five vendors supplying printed circuit boards average 98 non-conforming 
lots each or approximately 52 times more non-conforming lots than our overall 
average vendor who supplied 1.9 non-conforming lots. 

3. Due to these non-conforming lots, the average additional cost to process each 

lot inspected during this period was $3.24. (60280 t 18621) 

4. The average additional cost to process the lots supplied by the five printed 
circuit board vendors during this period was $18.02 per lot. (19680 v 1092) 

5. It cost 456% more to process material supplied by the five printed circuit 
board vendors than for our average vendor. (3.24 - 18.02 t 3.24) 


To further aid us in this study, we laid out a flow chart of the keyboard 
processes pertaining to the printed circuit boards. This gave us an overall picture 
of what is affected by the quality of printed circuit boards as every main step 
from the vendor all the way through to our final inspection. 

A meeting was then held with Production Engineering, Production and Final 
Inspection supervisors. It was determined that problems encountered in these areas 
relating directly to printed circuit boards appear to be minimal once the printed 
circuit boards have cleared through the Receiving Inspection Department. 

In the next phase of the study we reviewed the receiving inspection history of 
the boards in which 27 sequences are checked regularly. A Pareto Analysis was 
compiled showing the non-conforming defects from the five printed circuit board 
vendors for the eleven-month period. 


PARETO DISTRIBUTION ACCEPT/DEV. 



NO. OF 

% DE- 

ACCUMU- 

NO. 

% 

SPECIFICATION 

DEFECTS 

FECTIVE 

LATIVE % 

ACCEPT 

ACCEPT 

Hole Size 

165 

19.0 

19.0 

150 

90.9 

Board Dim. 

110 

12.7 

31.7 

96 

87.3 

Cond. Defects 

79 

9.1 

40.8 

23 

29.1 

Gold Thickness 

69 

8.0 

48.8 

31 

44.9 

Plating Visual 

66 

7.6 

56.4 

31 

50.8 

Gold Visual 

61 

7.0 

63.4 

27 

44.3 

PTH Thickness 

61 

7.0 

70.4 

10 

16.4 

Hole Location 

56 

6.5 

76.9 

54 

96.4 

PTH Visual 

29 

3.3 

80.2 

16 

55.2 

Bevel 

29 

3.3 

83.5 

11 

37.9 

Solder Mask 

29 

3.3 

86.8 

15 

51.7 

P/N & Artwork 

21 

2.4 

89.2 

20 

95.2 

Hole Registration 

18 

2.1 

91.3 

10 

55.6 

Gold Adhesion 

17 

2.0 

93.3 

1 

5.9 

Laminate Defects 

13 

1.5 

94.8 

5 

38.5 

Solderability 

10 

1.2 

96.0 

6 

60.0 

Vendor ID & U/L 

9 

1.0 

97.0 

5 

55.6 

Warpage 

6 

0.7 

97.7 

5 

83.3 

Plating Adhesion 

6 

0.7 

98.4 

0 

0.0 

Key Slot Location 

6 

0.7 

99.1 

3 

50.0 

Packaging 

5 

0.6 

99.7 

4 

80.0 

Material Thickness 

1 

0.1 

99.8 

1 

100.0 

Material Identification 

1 

0.1 

99.9 

1 

100.0 

Cu & Resist Removal 

1 

0.1 

100.0 

1 

100.0 

Pattern Registration 

0 

0.0 

100.0 

0 

0.0 

Scratches 

0 

0.0 

100.0 

0 

0.0 

Missing Circuitry 

0 

0.0 

100.0 

0 

0.0 


868 



526 

60.6 


Nine, or one-third, of these sequences resulted in 80.2% of the defects. In 
addition, those defects that were accepted on deviations were tabulated for each 
sequence. The top two defects, hole size and board dimensions, were accepted to 
"use as is," 90.9% and 87.3% respectively. 

Deviations were requested for a total of 868 non-conforming characteristics, of 
which 526, or approximately 61%, were accepted with no sort or rework required. 

This showed us that our product definition, i.e., what was specified on our prints 
vs. what we were accepting, varied to such a degree that none of the printed circuit 
board vendors could hope to achieve credibility. 



This brought us to a point where one ot the following steps was required 
regarding printed circuit boards: 

X. bo nothing - and spend in excess of $21 ,000 annually processing non- 
conformance reports. 

2. Eliminate inspection of printed circuit boards where accepted deviations 
exceeded 80%. 

3. Review and revise prints and/or specifications to reflect what we have 
been using 61% of the time. 

Management agreed that the first and second procedures were not acceptable and 
that we could revise, our prints to define the requirements correctly. As a result 
of this agreement, Engineering reviewed the data provided and made substantial 
changes in: 

Increased hole size tolerance range 

Increased board dimensions tolerance range 

Reduced gold thickness requirements 

In addition to the above, we are presently investigating the use of electrical 
testing of printed circuit boards to cull for shorts and voids. 

The final phase of this study was to visit the vendors and to survey the 
vendors' manufacturing system, process controls and gaging compatibility. We also 
reviewed their past quality record and in light of the revised specifications, we 
informed them we would not accept these defects as in the past. 

As a result of data obtained from this study, Engineering has revised its 
requirements. Through the quality survey a better understanding of the vendor's 
facilities and capabilities was obtained. 

Where our total non-conformance was 45.1% when we started our study, the figure 
for the first quarter of this year is 39%. Although this does not appear to be a 
significant improvement, if we look at the percentages of change in the "use as is," 
we find a very appreciable improvement. "Use as is" was running 60.6% when we 
started, while the figure for the first quarter of this year is 19%. This means 
that 81% of the non-conforming defects are being sorted, reworked or returned to 
the vendor and he is being billed for our time. Where we were initially returning 
3.9% of the boards we received, the figure now is 11.3%. This is the result of a 
concerted effort on the part of our Inventory Control and Purchasing Departments to 
assure that the majority of all products from rejected lots are returned to the 
vendor. We are only sorting and/or reworking sufficient quantities to meet 
immediate production needs and returning the balance for the vendor's action. Al- 
through the out-of-pocket cost to the vendor of this procedure is probably comparable 
to the rework and bill-back method, we feel that the vendor's awareness of his own 
in-plant operations is greatly increased. The vendor's Factory Production and 
Quality personnel arc aware of large inships of rejected material and vendor manage- 
ment is aware of large dollar credits which must be issued on this material. We 
believe, and our vendors have confirmed, that the need for and the interest level 
in implementing effective corrective action is heightened through this form of 
notification. Our early results seem to bear this out. 
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TOLERANCES VS. FABRICATION AND COST OF PC BOARDS 


Edmund E . Golonsbe 
Quality Assurance Manager 
Printed Circuit Division 
Methode Electronics, Inc. 
Chicago, Illinois 60656 


Printed circuit boards, unlike many purchased electrical com- 
ponents, are custom made to a specific dimension and design. 

There are no printed circuit boards that can be purchased as shelf 
items in the bare board state. Each board must be built to a 
specific design determined by the customer. Let me clarify what 
I mean by a shelf item. A shelf item would be the kind of an item 
that can be purchased from a dealer or catalog to be used in any 
number of units. Examples of shelf items are resistors, capaci- 
tors, integrated circuits, etc. 

Even when end items (calculators, televisions, etc.) are very 
similar, several manufacturers of those items will have a very 
different concept of what the printed circuit board should look 
like. For example, a few years ago, many plants manufactured hand 
held calculators. The basic calculator needed 10 keys for the 
functions. They also had to have a display for the numerals. 

Each manufacturer had these things but the circuit boards were 
quite different. 

The latest item made by many manufacturers are smoke detec- 
tors. Here, again, we have each printed circuit board performing 
the same function, but unique in its design and requirements for 
each company . 

This variation in design and specification can effect the 
cost of manufacturing the board and hence the price of the board 
to the customer. 

Because printed circuit boards are custom built and have 
many different requirements, they can be specified many differ- 
ent ways, there needs to be some standards to guide the designers, 
fabricators, quality personnel and assemblers. A set of standards 
for printed circuit boards has been developed by the I.P.C. 
(Institute of Printed Circuits)*. The I.P.C. is an organization 
of laminate processors, printed circuit board manufacturers and 
printed circuit board users (assemblers and equipment builders). 

In discussing cost related requirements for printed circuit 
boards, we are going to look at some of the tables and paragraphs 
of two of the IPC standards. 

IPC D300 - "PRINTED WIRING BOARD DIMENSIONS AND TOLERANCES 
TWO SIDED RIGID BOARDS." 

This specification covers dimensioning and tolerance limits 
based on industry capabilities. It provides five classes for di- 
mensional features to reflect progressive increases in sophisti- 
cation and cost of tooling, material and processing. 


♦Institute of Printed Circuits 
1717 Howard Street 
Evanston, Illinois 60202 
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IPC D320 - "PRINTED BOARD RIGID SINGLE AND DOUBLE SIDED, END 
PRODUCT SPECIFICATION." 

This specification covers end product workmanship and accept- 
ability requirements for all rigid single and double sided boards 
and provides classes for workmanship of a printed wiring board that 
reflects progressive increases in sophistication and cost for tool- 
ing, materials and processing. 

To give some idea of what is meant by more sophisticated tool- 
ing and materials and processing, here are some criteria that can 
effect the cost of manufacture: 

1. Hole Diameter Tolerance 

The tolerance could determine if a hole will be drilled 
or die perforated. 

2. Hole Location Tolerance from Datum 

May mean a more expensive laminate or require drilling 
in place of die perforating. 

3. Minimum Conductor Path and Minimum Space Between 
Conductors 

This will determine the method of placing the image on 
the board. Photo resist or screen, and in some cases 
the size of the panel. 

4. Registration Tolerance of Conductor Pattern to Holes 

The tolerance required will determine the size of the 
panel and the method of imaging. 

In our shop, the following rules apply: 

(a) +.015" registration tolerance will allow us to 
run a 36" to 38" long panel machine screen. 

(b) +-010 registration tolerance will restrict the 
panel length to 18” to 24". Machine screen or 
handscreen, or we can use a 36" panel then per- 
forate the indexing holes using an optical punch. 

(c) Less than +.010 to +.005 will restrict the 
length to 18". The image will have to be applied 
by Photo Resist . 

5. Workmanship Requirements 

(a) Smooth holes, no crazing or chips; holes must 
must be drilled 

(b) Smooth edges, no indication of undercut, etc. 

Board must be finished routed. 

(c) No marks, scratches, blemishes or measling on 
base laminate. Will not allow the removal of 
excess copper after etching. Will also require 
special handling and will increase the scrap ratio. 

(d) No pinholes, nicks or scratches in the conductor 
pattern. Would necessitate Photo Resist and touch 
up. Would definitely increase scrap ratio. 

The I.P.C. D300 places the various hole tolerances in five 
Classes, Class 1 with the broadest tolerances and Class 5 the 
tightest . 


The presentation in Table 2 is for nominal base material thicknesses up to and including 0.0625. (1,59). For nominal 
base material thickness over 0.0625 (1,59) and ±0.001 (0,025) to each tolerance. 


TABLE 2 - AS MACHINED; HOLES, SLOTS, AND NOTCHES 


FEA TURE 

CLASS 1 

CLASS 2 

CLASS 3 

CLASS 4 

CLASS 5 

0/0.032 Diameter 

+0.003 

-+0.002 

+0.001 

-0.002 

+0.001 

10.001 


(0,08) 

(0,05) 

(+0,025) 

(0,025) 

(0,025) 

0.033/0.063 Diameter 

10.004 

+0.003 

(-0.05) 

•0.002 

+0.001 

10.001 


(0,10) 

(0,08) 

(0,05) 

-0.002 

(+0.025) 

(0,025) 

0.064/0.188 Diameter 

+0.005 

10.004 

+0.003 

(0.05) 

10.002 

+0.001 


(0,13) 

(0.10) 

(0,08) 

(0,05) 

-0.002 

(+0,025) 

0/2.000 Slot or Notch 

+ 0.005 

±0.003 

+0.003 

+0.003 

(-0,05) 

+0.002 


(0,13) 

(0.08) 

(0,08) 

(0.08) 

(0,05) 

Ratio of Min. Hole 

2:3 or 

1:2 or 

3:10 or 

1 :4 or 

1 :5 or 

Diameter to Base 

66% 

50% 

30% 

25% 

20% 

Material Thickness 







[Tolerances indicate total spread and may be varied from the nominal to satisfy design requirements. For example: 

+ .004 (+0,10) 


» .005 +.006 (+0,13) (+0,15) 

-.003 -.002 (-0,08) or (-0,05) ] 

[When radii are not provided in slots or notches, there will be increased cost.) 


Let us see what can happen to increase the cost of the holes 
in a printed circuit board with different tolerances. 


Class 1) Unplated holes with a diametric tolerance of 

+.003 or greater, can be die perforated. Thus, 
many holes, slots and notches of various sizes 
can be perforated at the same time (most econom- 
ical for long runs) . 

Class 2) Unplated holes with a tolerance of less than 
+.003 will have to be drilled. Thus, the 
number of holes and the number of different 
hole sizes will have a bearing on the price 
of the board. 


Class 3) Unplated holes with a tolerance of +.001 may 
have to be reamed or at least given very 
special attention while drilling. Possibly 
require sorting. 


One of the unique characteristics of a printed circuit board 
that must be specified is the registration tolerance between the 
conductor land pattern and the holes. This may be done in a couple 
of ways. The simpliest way for the designer, if he disregards all 
other parameters, is to specify a minimum annular ring. The mini- 
mum annular ring is the amount of metal land left around the hole 
on the narrowest side. When the designer so specifies, sometimes 
it becomes very difficult for the fabricators to process the board 
as was intended at the time of quotation or layout. It could mean 
he has priced the board to a certain process and then discovers 
he has to go to a more lengthy process thus increasing the cost of 
manufacturing. This, of course, would also increase the price of 
the board. Even when the full land size is considered in relation 
to the hole size with the necessary tolerance to build the board 
economically . 

The designer may have to shave pads for electrical spacing- 
requirements, and unless the annular ring requirement is waved in 
that area, the quality control department is sure to (a) stop the 




job, (b) reject all product produced or (c) reject all incoming 
product. The registration tolerance can be also specified from 
centerline of hole to the centerline of the land pattern. This 
is the most straight forward way and the most easily measured 
and understood by the designers, fabricators and quality personnel. 


TABLE 3 - HOLE CENTERLINE TOLERANCES 


FEA TURE 

CLASS 1 

CLASS 2 

CLASS 3 

CLASS 4 

CLASS 5 

Greatest dimension from datum 
less than 6.000 

0.020 

0.014 

0.010 



DIAMETRIC TOLERANCE* 

(0,51) 

(0,36) 

(0,25) 

(0,15) 

(0,10) 

COORDINATE TOLERANCE 

±.007 


H'/. H 


±.0015 


(0,18) 

(0,13) 

(0,08) 

(0,05) 

(0,04) 

Greatest dimension from datum 
over 6.000 

0.028 

0.020 

0.014 

0.010 

0.007 

DIAMETRIC TOLERANCE* 

(0,71) 

(0.51) 

(0,36) 

(0.25) 

(0,18) 

COORDINATE TOLERANCE 

±.010 




±.0025 


(0,25) 

(0,18) 

(0,13) 

(0,08) 



*The stated tolerance is the permissible diametric movement of the true center of the feature in a circular area around the true 
position point regardless of feature size. 


One of the first considerations the designer must make is 
what registration tolerance can be allowed to keep the holes with- 
in the conductor pattern regardless of spacing and hole size re- 
quirements. Or what amount of breakout can be tolerated where the 
land size, because of electrical requirements, must be smaller 
than needed. If the requirement dictates there should be a mini- 
mum of .002 copper around each hole then he must determine the 
centerline to centerline tolerance required to maintain this an- 
nular ring, the minimum amount of copper (metal) around the hole 
on the narrow side, with a specified land diameter. 

In IPC D300 , there is a paragraph that deals with this con- 
sideration. This paragraph allows the designer to determine the 
size of land required where the hole size and positional toler- 
ances are known. 

9. LAND (TERMINAL AREA) SIZE. 

9.1 DESIGN FEATURE. The total accumulation of tolerances must be considered in selecting terminal size in relation to 
hole size. Since only one unknown is permissible, you must select either the hole maximum diameter or the terminal 
area size, and the class tolerance needed for hole position, line reduction, and terminal position. 

Example: Using 0.030'' (0,76) maximum diameter hole. 

A. Hole maximum diameter 0.030" (0,76). (Given) 

B. Hole positional tolerance regardless of feature size. Table 3, Class 4. 

C. Terminal area positional tolerance. Table 4, Class 4. 

D. Total possible reduction on the diameter due to processing. Table 6, Class 4. 

Note: Terminal area will be reduced in diameter just as conductors are reduced. 

E. Minimum edge of hole to edge of terminal area, i.e., annular ring. (As specified.) 

Minimum Terminal Area Diameter = 2 
(example) 

= 2 
= 2 


Minimum Terminal Area Diameter = 2 
Metric (example) 

= 2 



( 0 030 . 0.006 , 0.014 . 0.002 . .002^ 
\2 + 2 + 2 + 2 + ) 

< 0.015 + 0.003 + 0.007 + 0.001 + 0.002 ) 
0.056 inch 

( 076 . 0,15 . 0J6 . 0,05 , 0.05 
V2 + 2 2 + 2 + / 

(0,38 + 0,075+ 0.18 + 0,025+ 0,05) 
1,42 mm 





Many times, a designer, through necessity, will have to de- 
sign a board with various size pads and even then many pads may 
have to be cut or flattened. In this case, he may want to specify 
a Class 1 registration tolerance to take advantage of a low piece 
part price and knowing breakout will occur will probably want to 
limit the expected breakout to the workmanship standard IPC D320 
TABLE 6 - MINIMUM ANNULAR RINGS. 

Land minimum annular rings shall be in accordance with Table 6. [NOTE: When land diameters are reduced by 
trimming, the master drawing shall stipulate where minimum annular ring shall be measured.] 


TABLE 6. - MINIMUM ANNULAR RINGS. 



CLASS 1 

CLASS 2 

CLASS 3 

CLASS 4 

CLASS 5 

UNSUPPORTED 

75% 

90% 

No 

0.010 

0.015 


Enclosed 

Enclosed 

Breakout 

(0,25) 

(0,38) 

SUPPORTED 

75% 

90% 


0.003 

0.005 

1 

Enclosed 

1 

Enclosed 

Tangent 

(0,08) 

(0,13) 


Workmanship standards probably have the most variation of in- 
terpretation between the board fabricator, designer and/or product 
engineer and quality control. 

IPC D320 has tried to classify various visual attributes such 
as: cracks, chips and fibers (Table 3): 


TABLE 3. - CRACKS, CHIPS & FIBERS FROM INSIDE EDGE [Maximum] 


EE A TURE 

CLASS 1 

CLASS 2 

CLASS 3 

CLASS 4 

CLASS 5 

Cracks & Chips 

0.040 

0.020 

0.010 

0.005 

0.005 


(1,02) 

(0,51) 

(0,25) 

(0,13) 

(0,13) 

Fibers 

0.030 

0.020 

0.010 

0.005 

0.005 


(0,76) 

(0,51) 

(0,25) 

(0,13) 

(0,13) 


And visual inspection in the conductor pattern (Table 5): 


TABLE 5. - ALLOWABLE VISUAL IMPERFECTIONS IN THE CONDUCTOR PATTERN (Maximum] 


ATTRIBUTES 

CLASS 1 

CLASS 2 

CLASS 3 

CLASS 4 

CLASS 5 

Conductor Width- 
Nicks & Pinholes 

50% 

35% 

25% 

20% 

15% 

Conductor definition* 

0.010 

(0,25) 

0.010 

(0,25) 

0.005 

(0,13) 

0.005 

(0,13) 

0.003 

(0,08) 

Roughness, (Crest to Trough) 

0.015 

(0,38) 

0.010 

(0,25) 

0005 

(0,13) 

0.005 

(0,13) 

0.005 

(0,13) 

Scratches (Depth) 

50% 

35% 

25% 

20% 

15% 


'As measured to the production master. 


It should be understood the more stringent the visual attrib- 
utes, the more sophisticated the tooling must be. Sometimes it be- 
comes a sorting problem. The only recourse the fabricator can 
have in this case is to manufacture enough product to cover the 
scrap. It is possible that very close tolerances are necessary 
and the board fabricator will then use the proper tooling and safe 
guards to produce the type of product specified, and he will make 
sure the price will insure his making a profit. These boards can 
be very expensive. 

Remember unnecessarily tight tolerances can increase the cost 
of the piece part, sometimes as much as 10 times the expected cost. 
Do not be fooled by the company that bids low because the close 
tolerance is hidden or overlooked. This company will often re- 
quest waivers, an increase in price or at best, will increase the 
price on the next order. 










In some cases, tooling will be made only to find it is im- 
possible to manufacture to the expected tolerance. it is better 
to understand the capabilities of the material, process and tool- 
ing before committing the order. Clear and easily understood 
drawings and specifications are necessary to allow a reasonable 
and fair quotation. If possible, a meeting with the printed 
circuit board fabricator can be very helpful. 
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MANUFACTURING SYSTEMS CONTAINING CSP AND LOT PLANS 


M. Zia Hassan, Ph.D. 

Illinois Institute of Technology 
Chicago, Illinois 60616 

A* C. Endres , Ph.D. 

Motorola Communications Division 
Schaumburg, Illinois 60196 

INTRODUCTION 

A great deal of attention has been given to the problem of con- 
structing "optimal" single or interrelated Lot-By-Lot (LBL) saimolina 
plans. Corresponding efforts directed at the construction of "optimal" 
single and interrelated Continuous Sampling Plans (CSP-1) have been 
much more restricted. The paper develops a dynamic programming model 
for determining "optimal" sequences of interrelated LBL and CSP-1 Plans 
with specifiable process averages (with associated control costs) and 
limits on average outgoing quality. 

MODEL FORMULATION 



Figure 1 Single Manufacturing Stage with LBL and CSP-1 Inspection Stages 

Figure 1 depicts the model of interest. The vendor's process 
average pj. is assumed specifiable with an associated per unit cost of: 

k 4 <p i' = pj + 6 


where y and <$ may be estimated from the vendor's pricing information. 


At stage one the vendor's material (submitted in lots of size N-^) 
is sampled via an LBL rectification plan whose parameters (n,, c^) 
must be determined by examining the LBL expected cost function. This 
cost function is comprised of the sum of the expected per unit costs of 
inspection K^: 


K Si = k 21 ATl1 


and 


ATI 1 = n 1 


(N. 


.‘■T 1 ( 


i - P e 

x=o — 


'i p i 


(n i p i ; 


x: 


repair 


where kji is the average per unit inspection cost and the expected unit 
cost K r1 where: 

k n p- ATI, 

ki = r 


This work was partially supported by the Office of Naval Research, 
Contract N00014-76-C-0841 Task NR 042-360. 






where is the average unit repair cost, therefore the total exnected 

inspection station cost is: 

ATI, 

TUISC 1 = — (k U Pi + k 12>' 

The material then proceeds to be progressively manufactured at 
Stage two where the fabricated unit reflects either vendor or manufac- 
turing defects with probability 

AOQ 2 = AOQ 1 + pJ, - p 2 AOQ 1 = AIO 3 

where p 2 is the selected process average at manufacturing stage two 
with expected per unit cost: 



with a 2 and R 2 being constants associated with the fabrication process. 

The fabricated units are then inspected via a CSP-1 plan at stacre 
three with required AOQ £ AOQL 1 , the specified Average Outgoing Quality 
Limit. The expected per unit costs of inspection and repair correspond- 
ing to Stage one are now: 

K S3 = k 23 AFI 3 

and K r3 = k 13 p 3 AFI 3 

tuisc 3 = afi 3 (k 13 p^ + k 23 ) 

Dodge ^ has demonstrated that for CSP-1 


AFI 3 = 


u 3 + f 3 v 3 
u 3 + V 3 



Since the TSUC expression is the sum of four individual cost 
expressions it is convenient to decompose the problem via Dynamic Pro- 
gramming ) . The procedure essentially solves for the decision vari- 
ables, sequentially, stage by stage, in lieu of simultaneously. The 
additional notation required for this formulation is (3) 



expected per unit cost at stage i = 1, 2, 3 


f . (AOQL) = minimum cumulative expected system unit cost throuqh stage i 
as a function of the Average Outgoing Quality Limit for Staqe 
i = 1 , 3 and for i = 2, AOQL = AOQ. 

Q ± = r ± + f i _ 1 (AOQL) 

= decision variables at stage i 

therefore 

f. (AOQL)= Min [Q. ] = Min [r. + f, , (AOQL) ] 

1 D i D i 

Stage One Minimization Procedure 


The solution for p-j., n^, and c^ for a given AOOL^ output is to 
establish a grid of feasible P-j- > ' values and solve at each crrid pointy, 
p ' , ' for the n^ and c^ which minimize ATI-^ by the Dodge-Romig procedure . 
TnJ resultant expected per unit cost for each grid point p^.'is: 


= — r~r + <$ 
P- 


ATI. 


(k 


Ij 


11 p ij 


+ k 12> 


and D 1 = p^, (n 1/ c 1 ) 

This procedure is repeated for each successive grid point pi.' until 
the value os pi . 1 is found which minimizes the expression for the 
given AOQL value. The procedure is then repeated across a correspond- 
ing grid at AOQL.. . values with the best p' . ' , n, , c.. being solved for 
each AOQL^j grid 1 -^ value. ^ 

Stage Two Minimization Procedure 


At stage two 

01 2 

Q 2 = r 2 + f 1 (AOQL lj ') = + B 2 + f 1 (AOQL 1 .') 

and 

°2 - p 2 

We now set up a grid of AOQL 2 _: values and solve for the p 2 value which 
minimizes Q 2 for each AOQL 2 j. Note that since 

aoql 2 . - p' . 

p' - AOQ ' AOQL,. - AOQ ' 

P2 " tF - AOQ^) " ~n~ - 1 AOQ Y j ~ l "i 3 - 

Hence, we need only pick an A0QL2j grid value, recall the first AOO-^ ' 
value which resulted from Stage one minimization, and solve directly 
for p 2 . This procedure is repeated for all feasible combinations of 
AOQL 2 . and AOQij ' until the minimum value of Q 2 is found for the current 
AOQL 2 ^ value. The previous AOQL 2 j value is changed to A0QL2j+i and 
the procedure continues until the largest feasible A 0 QL 2 j value is 
completed. Each combination of AOQL 2 .. and AOQ^ ’ so found is saved. 

Stage Three Minimization Procedure 

At stage three 

r 3 + f 2 (AOQL 2 j ' ) = AFI 3 (k 13 AOQL 2 j'+ k 23 ) + f 2 (AOQL 2 j ' } 


1 3 / f 3 




with specified AOQ 3 <_ AOQL' . 

Now Resnikof f ^ has shown that for 
AOQL 2 = AIQ 3 > AOQL ' 

AFI 3 will be minimized when 


, = 1 " AIQ 3 

X 3 AIQ 3 - AOQL r 

and 

AIQ - AOQL' 

f = — 

AIQ 3 - AOQL' + AOQL' (1-AIQ 3 ) W 

where 

(1-AIQ 3 ) 

Q = AOQL' - AIQ 3 

which yield 


AFI = 


AIQ 3 - AOQL' 
AIQ^ 


However if AIQ 3 <_ AOQL' , 

AFI 3 can be made as close to zero as is consistant with desired protec- 
tion against "spotty quality" by choice of arbitrarily small f 3 . For 
purpose of simple demonstration, we shall always set f 3 = 0 for AI0 3 <_ 
AOQL' which of course yields AFI 3 =0. 

We can proceed, therefore, by using the input grid of AOOL^j ' 
values and set either AFI 3 = 0 if AOQL 2 j * <_ AOQL' , or 1 


AF! 3 = 


AOQL 2 . ' - AOQL' 
AOQL^ 


if AOQL 2 j ' > AOQL' and thereby, determine the values of AOQL 2 j ' , i 3 , f 3 
which yield the minimum combined cost for the three stage system, i.e. 
the minimum of Q 3> Once these values are determined we can backtrack 
through stages two and one to determine their optimal values by recall- 
ing the decision variable values which lead the optimal values for 
AOQL 2 j ' and AOQL]_ j ' , respectively. Finally, it is to be noted that 
since J we have used grid search in conjunction with a complex TSUC 
response function, the optimal values may yield only a local instead of 
a global minimum. 


EXAMPLE PROBLEM 


Table 1 contains the specific values of the example problems para- 
meters. Table 2 and 3 contain the solutions determined for the initial 
grid size of .1 and final grid size of .005, respectively. (Inter- 
mediate solution for grid sizes of .05, and .01 have not been shown 
because of space limitations.) Note that the required AOOL 3 ' for the 
system is 10%. 



Table 1. Example Problem’s Parameters (AOQL' 3 = .10) 



Table 2. Solution for Grid One Size = .1 



Table 3. Final Solution for Grid Size = .005 


Stage No. 

Stage Type 

Decision Variable 0* = 

= Minimum Cumulative 
Expected Unit Cost 

0 

Dummy 

p* = .035 

- 



n* = 400 


1 

LBL 

c* = 0 

1.35 

2 

Mf g . 

p^* = .135 

9.75 

3 

CSP-1 

i* = 24 f * = .0096 

11.76 


Figure 2 contains the AOQ curve for the CSP-1 plan selected for 
Stage 3. It is to be noted that in accordance with the previously 
developed AFI formula for A0QL 2 j ' > AOQL ' = .10 that AFI 3 = (1.35 ->.10)/ 
.135 = .259 and that indeed A0Q 3 = AOQL' = .10 at the average incomina 
quality level. of .135. 
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SO YOU WANT TO DO A SUPPLIER SURVEY IN THE WORST WAY 


PART 1 1 


Richard A. Maass 
Abbott Laboratories 
North Chicago, Illinois 


After a successful two year run on the ASQC circuit (15 performances) the 
Vendor-Vendee Technical Committee is returning to center stage by popular demand with 
Part II of its morality play of "So You Want To Do A Supplier Survey: - In The Worst 
Way" . 


The play, presented at the 30th ATC in Toronto, has the professional survey team 
from Exact Products surveying the highly competent Quality Widgets. The other half 
of the play had Far Out Enterprises surveying Whoopee Widgets. The latter survey 
was both hilarious and horrendous as we watched the two teams out-do the "F Troop" 
without really trying. The actors portrayed some of the most common evaluation 
mistakes and the audience roared with laughter because most of us recognized these 
boners all too well . 

After the show the audience suggested that roles be reversed. Let Far Out 
Engerprises deal with Quality Widgets and Exact Products survey Whoopee Widgets, 
after all, this combination probably represents true to life conditions more 
accurately. 

That is what Part II is all about, so come and enjoy the show. See what happens 
to Sam, the crusty old Quality surveyor, the four martini lunch, George's and his 
purchasing agent kickback, and "Does Jungle Jim the Vice President ever learn what 
a quality survey is?" 

The show should be fun and we might even learn a little on how to handle unique 
situations. We all need it even if we all work for Exact Products, because we are 
always dealing with Far Out Enterprises or Whoopee Widgets. 
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THE UNSOPHISTICATED QUALITY VENDOR 


W. H. Anderson 
General Electric Company 
Philadelphia, Pennsylvania 


By definition "The Unsophisticated Quality Vendor" is one, through lack of 
knowledge, that has not implemented an effective quality system. 

The Vendor-Vendee Technical Committee, as one of its long range projects, 
is exploring ways and means to convert "The Unsophisticated Quality Vendor". 

Come and join with us as this program is designed for you, the audience, to 
participate in the search for the answers and determine what are the Society's 
responsibilities in this endeavor. 

Our guests will be: 

Mr. John Song: President, Magenta Plastics. John represents the small vendor. 
He is unique in that he has recognized the advantages that can be gained with an 
effective quality system. We can all benefit from his trials and tribulations in 
the Quality arena. 

Mr. Don McNeill: Chief, Quality Assurance Telecommunications and Electronic 
Branch Canadian Government. Don has had considerable overseas experience both in 
Europe and Australia. Don may be able to give us some insights on how the problem 
is approached on the other side of the oceans. 

Mr. Harry D. Greiner: President Management Systems Analysis. Harry's long 
career as a quality manager for Budd cast him in a role of a vendor, as a Product 
Assurance Manager for RCA he was a vendee. In his present capacity of a manage- 
ment consultant he works both sides of the street. Harry's technical expertise 
is welcomed . 

Mr. William Anderson: General Electric Re-Entry & Environmental Systems 
Division will act as host. 

You, the audience, will be afforded the opportunity to express your opinions 
and ideas, challenge the guests and ask pertinent questions. 

This is intended to be a working session. The ideas generated at this session 
will reflect in a future "How To Handbook" planned by your Vendor-Vendee Technical 
Committee. 
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DON'T MAKE THEM LISTEN - MAKE THEM DO 


B. Or jan Alexander son, M. Eng. 
Management Consultant, EKAB 
Espedalen 8925 
S -69100 Karlskoga, Sweden 


THE PROBLEM 

During the last years many Managing Directors have heard the following state- 
ment from their QC Managers: 

"We must develop a more systematic approach to our quality problems. We will 
have to design a better system for calculating our quality costs, a written procedure 
for identification of accepted and rejected batches, a better way of evaluating our 
vendors, and so on. This is all necessary for many reasons, but first of all - our 
customers demand it." 

Let us try to imagine what thoughts these words bring about in the mind of the 
Managing Director: 

"This will cost at least Z 40,000 to 50,000." 

"What's wrong with the companies that we buy from? Besides, that is not his 
business, he is supposed to inspect the end product." 

"Quality costs? We have had very few customer complaints the last 6 months." 

"1 don't: believe that our customers realty demand this. They will continue to buy 
from us as long as we deliver good products." 

"Nobody will ever read the written procedures he is talking about. They will 
just end up in a dusty binder somewhere." 

So the Managing Director will probably answer his QC Manager something like 
this : 


"Yes, I agree that we should woidc sy sternal icly in order to maintain our good 
quality reputation. But at the moment we cannot allow an increase in our over-heads, 
so I must ask you to achieve your objectives step by step with your present re- 
sources . " 
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ANALYSIS 


Obviously there is a lack of knowledge on the part of the Director, He does not 
know what we mean by quality costs or vendor evaluation. He is not aware of the 
change in customers' interest, from inspecting the products to demanding an appro- 
priate Quality Assurance System. He does not see, that a systematic quality control 
can save him money and make sure that old customers staj^ and new ones turn up. 

Let us not put the blame on the director, at least not all of it. Nothing is ever 
achieved by blaming people (and definitively not by blaming your boss*.). It is much 
more useful to try to find, out what we could do about the situation. 

It seems to be a very trivial question of information. A couple of hours would be 
enough to teach the whole top management the concept of modern Quality Assurance, 
and convince them that there is money to earn. And during my first years as a 
management consultant, this was what 1 was hired to do on many occasions. I had a 
fine information material, developed by the Swedish Association for Metalworking 
Industry. I had a true belief in what I was preaching. And I considered myself to be 
an excellent lecturer*. 

In the beginning I also had the feeling that I was completely successful. The 
managers I spoke to listened carefully, and they always told me after my presenta- 
tion, that they had found it extremely interesting. So I got the impression, that 1 
was doing a very important job, that brought the QA concept forward within Scan- 
dinavian industry. 

But after some time I decided to do some follow-up on my previous assignments. 

I called my clients to hear what had been done as a result of my presentations. The 
result of this investigation was most disappointing. Very little had been achieved, 
in many cases nothing had been done at all. Immediately after T had left the company, 
everything went back to normal again. 

The only explanation I have been able to find - and it is confirmed by my recent 
experiences - is that my information was understood only in a pure intellectual 
sense. And that is not. enough'. The managers must understand the problems with 
their feelings too, they must feel the problems on their bodies before they will put 
them higher up on their priority list. 


SOLUTION 

Unfortunately we cannot make the Managing Director QC Manager for a year or 
so, even if that of course would make him sec that some of our demands are right. 
No, we must find a method, similar to accelerated life -length -testing, that enables 
us to create true involvment in a short period of time. 


The following program has proven successful on several occasions, in companies 
of different sizes and branches. 

Target group: Top Management and other key persons. At least 10, at most 20 
people attending. 

Time and place: Two days in a quiet environment outside the company. 

Program: Principally the conference or seminar consists of three steps , namely 

- the policy step 

- the problem solving step, and 

- the planning step . 

All three steps are taken as group works or syndicates. 

The policy step. The participants get a number ( 15 - 20 ) of written statements 
about quality, the quality situation in the company, the organization of the quality 
work, etc. The}' have to state their opinion about all the statements, and they have 
5 possibilities to choose between: Agree totally, agree partly, uncertain, disagree 
partly and disagree totally. 

When everybody has made up his mind, people go out into smaller groups, i. e. 
5-7 persons, and now they shall try t.o agree on a common view on each statement. 
The discussion that takes place in these groups leads to a concious evaluation, that 
will make a base for a policy. 

The problem solving step. In this next exercise the participants are given two 
alternative plans of action for each of 5 - 7 important areas within the field of QA . 

It should be areas where some action is necessary, and where it is known to exist 
different opinions about what should be done. The two action plans shall be extremes 
in two different directions. The participants are asked to distribute 10 points 
between the two plans for all of the descibed areas. You can for example for one 
area give 10 points to plan A and 0 points to plan B, meaning that you totally agree 
with plan A. For another area you can, if you find the two plans equally good ( or 
equally bad ), split the points 5-5. 

Also in this exercise you try to come up with a group solution that everyone in the 
group can agree upon. The result is often, that the explanations the groups give to 
why they have decided on a certain distribution of points, is really a third plan of 
action, which is a combination of the best in the two original plans. This means, that 
you have reached an agreement on how to solve the problems within that particular 
area . 


The planning step . At this stage, the participants usually are rather excited 
about the plans of action they have agreed upon. But this is only a positive attitude. 
It is now necessary to evaluate and compare the benefits and the drawbacks of the 


plan chosen for each area. Only when this is done you have got a true motivation to 
carry out the plan. If you merely have got a positive attitude, there is always a risk 
that your enthusiasm fades off when you realize the efforts you have to make. 

So this is why the third step is so important. The groups must draw the con- 
clusions of their plans of action, i. e. what resoui'ces they have to allocate. 

CONCLUSION 

The ke3^ elements of the program are: 

1 . You don't tell people what has to be done. You give them the problems and let 
them come up with the solutions, not in detail but the princips . It is in the following 
project work that you tackle the details with the help of your specialists. And it is a 
great advantage for the specialists to know, that they can always turn to the managers 
for guidance - they are familiar with the project. 

2. The group exercises are designed in a way that makes everyone qtiantify his 
view before he gets the opportunity to listen to other people's opinion. Nobody has 
got a chance to be too tactical, everybody must be prepared to explain his opinion 
about a statement or a plan of action, 

3. The participants do not only create a plan on how to tackle the problems. They 
also make a thorough evaluation of the needs of resources to carry out the decisions 
that are made, 

4. The conference is held in an undisturbed environment, and the time involved 
is reasonably long. Do not try to achieve a result in a shorter time, or with the 
program divided into shorter periods. It takes some time to clear one's mind from 
other problems . If you switch too often you may find yourself spending most of your 
time getting rid of the last problem you discussed. 

The companies that have used this program have found, that they did not only solve 
some problems, they also created a basis for future problem solving. It is a general 
observation, that people who have achieved something together get along fine in the 
future. It is a true asset to an organization to have links of this informal kind, built 
on personal appreciation antitrust. 
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GENERIC QUALITY STANDARDS - A CANADIAN APPROACH 


Charles A. Mills, F.I.E.E., P.Eng. 
Chairman 

Steering Committee on Assurance Sciences 
Canadian Standards Association 
Rexdale, Ontario, Canada 


INTRODUCTION 

During the past few years, activities in industry, standards 
organizations, legislatures, etc. , made it clearly evident that a co- 
ordinated effort was required with respect to the Quality field in 
both the product and service industries. The proliferation of quality 
documents, in an unco-ordinated way, was leading to a potentially 
chaotic situation. This was particularly critical to the small supplier 
serving a variety of industrial fields. 

The Canadian Standards Association (CSA) recognized this problem 
and initiated a Steering Committee on the Assurance Sciences to review 
the needs, in this field, for voluntary consensus standards. The term 
Assurance Sciences was used in lieu of Quality Control or Quality 
Assurance in an endeavour to improve the communications between the; 
various disciplines involved. 

The Standards Council of Canada accredits the Standards Writing 
Organizations (SWO's). Through the Council, agreements are reached on 
which SWO will prepare consensus standards in their particular area of 
expertise. In the Quality field, CSA has now been recognized as the 
SWO responsible for the preparation of an integrated system of Assurance 
Standards for promulgation as National Standards of Canada. 

In developing this integrated system, it is important to ensure 
that the 'needs' of the other accredited Canadian SWO's are known. It 
is also important to harmonize these activities with the various 
international activities in this field. 

ORGANIZATION 

In order to meet the overall requirements, CSA has initiated the 
Steering Committee on Assurance Sciences to guide and co-ordinate the 
activities of the various technical committees and other committees 
involved in the development and application of a system of Assurance 
Standards . 

Steering Committee members are selected for their individual 
expertise. However, in order to maintain a balanced approach, every 
endeavour is made to balance members between the three categories: 

1. Supplier organizations. 

2. User organizations. 

3. Regulatory bodies and general interest, including academic. 

In most cases the members representing the supplier and user organiz- 
ations will be filling a dual role since most suppliers are also major 
users of material and/or services, and most of the user organizations 
are suppliers of services. 

The activities of the Steering Committee are supported by: 

1. Secretariate, provided by CSA. 
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2. Executive Committee. 

3. Co-ordinating Committee. 

4. Consultative Committee, 

The CO-ORDINATING COMMITTEE has been formed as a staff aid to the 
Steering Committee. The membership of this committee will comprise 
the representatives of the accredited Canadian SWO's, certain of the 
Government Procurement Agencies, and other selected bodies. Representa- 
tives of the CO-ORDINATING COMMITTEE are expected to reflect the 
'needs' of their respective agency or body to the Steering Committee. 
They are also expected to provide visibility to their respective body 
on the activities of the Steering Committee. In this way, we hope to 
ensure a co-ordinated approach which will meet the needs of all. organi- 
zations wishing to use the ultimate standards. 

The CONSULTATIVE COMMITTEE is a small committee of experts in the 
Assurance fields who can act as consultants and advisors, assist with 
interpretations, and provide other related guidance, on request, to 
product or service technical committees of CSA and the other SWO's. 

OBJECTIVE 

THE PRIMARY OBJECTIVE OF THE STEERING COMMITTEE ON ASSURANCE 
SCIENCES IS TO DEVELOP AN INTEGRATED SYSTEM OF STANDARDS TO PROVIDE 
REASONABLE ASSURANCE THAT PRODUCTS AND SERVICES MEET THEIR INTENDED 
REQUIREMENTS, IT IS THE INTENT THAT THESE STANDARDS CAN BECOME NATIONAL 
STANDARDS OF CANADA. 


INTEGRATED SYSTEM 

It has been agreed that the integrated system will basically 
comprise four types of documents: 

1. Generic System Standards. 

2. A Management (Administrative) Guide. 

3. Technological Standards. 

4. Product or Service Standards. 

The Steering Committee on Assurance Sciences is responsible for 
the development of the first three categories of these standards. The 
fourth category - Product or Service Standards - falls under the 
jurisdiction of the Steering Committee responsible for a particular 
product or service in CSA's case, or under the jurisdiction of the 
appropriate SWO in other cases. 

ASSURANCE SYSTEM STANDARDS 

It has been recognized that modern industrial management, normally, 
determines and measures the effectiveness of its organization by means 
of quantitative objectives. Successful industrial management is a 
creative process, which can result in a variety of solutions to common 
objectives. Each solution must match the resources available within a 
particular situation to the objectives. 

One of the key objectives for all organizations is the need to 
assure the integrity of the product or service provided. In some 
areas, this objective has arisen through the requirements of specific 
contracts. In general, however, this demand has arisen through indus- 
trial conscience and competition, plus consumer awareness. 
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Recognizing the above, it is the intent to develop the System 
Standards in terms of objectives. By defining the assurance requirements 
in this way, industrial management will be able to apply their resources 
in the most efficient manner applicable to their product or service. 

It must be recognized that System Standards can have a dual purpose. 
Firstly, they can provide reasonable assurance, to management, and 
certifying agencies, that products or services meet their intended 

requirements. Secondly, they can form part of the contractual require- 
ments for a particular product or service. The complexities of system 
required for each of these purposes need not be the same. 

Therefore, it has been decided that standards will be issued 
covering the different levels of system complexity. These comprise: 

1. Quality Assurance Standard - a Quality system standard 
applicable to an organization having responsibility for a 
product or service from its conception to its application, 
including installation and dismantling where applicable. 

2. Quality Control Standard - a Quality system applicable to 
organizations providing complex or critical material or 
services to the customer's conception and design documentation. 

3. Inspection System - a Quality program applicable to organiz- 
ations providing repetitive and/or simple material or services. 

These standards, defining 'what' the requirements are for each 
level of complexity, can readily be used as the basis of measuring the 
effectiveness of the Quality program. It can form the basis of certify- 
ing the Quality program of an organization to a known level for a 
particular product line. The technologies and methodology used will 
depend upon the complexity of the product line. The basic requirements 
of the standards, however, will be based upon those elements of the 
Quality Assurance profession which are common to all product lines. 

MANAGEMENT GUIDE 

A Management Guide is in the process of preparation, taking the 
form of a generic 'Guide to the Interpretation and Application of 
Quality Assurance Standards'. 

This document is intended to provide visibility to the Chief 
Executive Officer (CEO) and other senior members of management, outside 
the Quality profession, on 'why' the various requirements exist within 
the system standards. The Guide will be based on the Quality Assurance 
system, since any of the lower levels of system complexity will comprise 
selected elements of the Quality Assurance system. The program covered 
by the guide will cover the total life cycle of the product or service. 
The combination of the Guide and the applicable system standard should 
provide the CEO with the answers to the 'what' and 'why' of an Assurance 
system. This will permit him to exercise his management prerogative 
in assigning responsibilities for the implementation of various aspects. 
It will also provide him with information on which to measure the 
effectiveness of the management of his Quality program. 

TECHNOLOGICAL STANDARDS 

Technological Standards are required in each of the major elements 
of Quality expertise. Each standard will cover a particular area of 
expertise or technological requirement of the system standard. 

These standards will provide the 'how' to implement the various 
requirements of the system standard. It will be essential that we 
ensure that these standards do not inhibit management creativity in 
matching the resources available to the product and/or service need. 


PRODUCT /SERVICE STANDARDS 


Product or Service Standards will be prepared by specialist 
groups, product or service, as required. These standards may incorpo- 
rate or reference the use of the management and technological standards 
as necessary to suit the peculiarities of the particular product, 
service or application. 


PLANNING 

One of the supporting requirements we set ourselves was to not 
add to the proliferation of Assurance standards where an existing 

standard had been demonstrated as meeting the requirements of the 
Steering Committee. This implied a fairly comprehensive review of the 
existing generic standards as part of the planning operation. 

The initial step in the planning of the Steering Committee 
activities was to develop a Function Tree covering the various elements 
of an Assurance system. 

The Function Tree is a well known system analysis tool. The 
prime function is sub-divided into its various supporting functions. 

Each supporting function in turn is sub-divided into lower level 
supporting functions. By repeating this process until the basic 
Quality elements are identified for each function, it is possible to 
develop a system diagram providing full visibility on the various 
elements of the system. 

Having developed the Function Tree for the Assurance system, 
indexes for various standards issuing organizations, e.g. Canadian 
Standards Association, Canadian Government Standards Board, Department 
of National Defence, Department of Defense, American National Standards 
Institute, British Standards Institute, etc., were examined to identify 
those generic standards already in existence for the various elements. 

As the system of standards develops, revisions will undoubtedly 
be required to the initial Function Tree. The Function Tree will, 
however, still provide a ready reference to ensure co-ordination of 
the activities of the various technical committees. 

PREPARATION OF STANDARDS 

It has been recognized that the writing of standards by large 
committees is fraught with the same hazards that apply to designing a 
product by a large committee. Therefore, we have instituted a series 
of small Task Forces to carry out the preparation of the initial drafts. 
In certain cases these Task Forces report to technical committees, and 
others to the Steering Committee. 

When the first drafts have been developed, technical committees 
and technical sub-committees will be initiated to review the drafts 
and to act as guides to the further development by the Task Forces of 
the particular standards. 

It has been stressed to each of the Task Forces that it is not 
the intention of the Steering Committee to develop new standards where 
existing standards suffice. If an existing standard meets the needs 
in its existing form, or with minor modifications, steps are being 
taken to implement some form of Canadian Notice of Adoption which will 
indicate the identification and revision of the document adopted, and 
its degree of acceptability including any minor deletions and/or 
additions. This notice of adoption will be subject to periodic review. 
If the referenced document is changed by the originating body, the 
revised document will be subject to review with respect to its adoption 
into the Canadian system. 



STATUS 


Currently, technical committees are active in the following 
areas : 

1. Guide to the interpretation and application of Quality 
Assurance systems. 

2. Application of statistical methods. 

The Technical Committee on the Management Guide has reviewed the 
second draft of the Guide. The third draft is currently in the process 
of preparation. 

The Technical Committee on Statistical Methods has a number of 
sub-committees reviewing the various statistical fields to determine 
the extent of its activities. This Technical Committee has been 

closely harmonized with the Canadian National Committee for TS0/TC69. 
Both Technical Committees have the same Chairman, plus a number of 
common members. 

In addition to these Technical Committees, Task Forces have been 
initiated to review the requirements in the following areas: 

1. Assurance Systems. 

2. Design Integrity (Assurance). 

3. Production Assurance. 

4. Metrology. 

5. Software Assurance. 

6. Quality Audit. 

The status of the work within each of these Task Forces tends to vary. 
However, each is charged with the responsibility for: 

1. Reviewing the existing standards. 

2. Assessing the need for new or modified standards. 

3. Making recommendations with respect to standards. 

4. Making recommendations with respect to the Technical Committee 

membership. 

5. Preparing a Statement of Work for the Technical Committee. 

Each of these Task Forces will be utilizing segments of the Function 
Tree to indicate their requirements for standards. 

As standards are prepared or notices of adoption are prepared, 
they will firstly require balloted approval by the Technical Committee. 
The approved documents are then subject to review and approval by the 
Steering Committee on Assurance Sciences who in turn are responsible 
for submitting these documents to the Standards Policy Board for 
submission to the Standards Council of Canada. 



HARMONIZATION 


The broad spectrum of membership within the Technical Committees 
should ensure the harmonization of the various needs. A similar 
breadth of industrial and service expertise in the Steering Committee 
on Assurance Sciences, supported by the Co-ordinating Committee, 
should ensure that the various standards developed harmonize into a 
national integrated system. 

It is essential that the Canadian system of Assurance standards 
is harmonized with the International systems - ISO, IEC, etc. Currently, 
our harmonization is concentrating on three organizations - ISO, IEC, 
and CEE. In these areas, representatives of the Canadian National 
Technical Committees are members of the CSA Technical Committees. In 
at least one instance the same individual is the Chairman of both 
Committees. Currently, ISO/TC69, IEC/TC56 and IEC ' Q f are involved. 

As an aid to international harmonization between the English 
language standards, formal liaison contact has been initiated between 
the Staff Secretariate of ANSI, BSI , and CSA as well as direct contact 
between the Chairmen of the corresponding Steering Committees. It is 
hoped that this type of liaison will serve to at least reduce the 
proliferation of standards in the English language. 

CERTIFICATION 

Whilst not forming part of the terms of reference of the Steering 
Committee on Assurance Sciences, it has been recognized that some form 

of certification of organizations to appropriate systems standards is 
required, both on a national and international basis. Steps have been 
initiated to implement this type of activity at the National level. It 
is hoped that international industrial and commercial reciprocal arrange- 
ments similar to that used by NATO, can be developed for commercial and 
industrial applications. 


CONCLUSIONS 

We believe that with our approach to the development of an inte- 
grated system of National Standards, we can provide an integrated generic 
system meeting the 'needs' of Canadian industry, which will harmonize 
with the international standards. 



UNIDO STANDARDS AID— THE NIGERIA EXPERIENCE 
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UNIDO.' AID 

Since its establishment in 196?, the United Nations Development Or- 
ganization (UNIDO) has engaged' in a continuing and increasing program of 
technical assistance to developing? countries. Among many areas of as- 
sistance, the development of national standards organizations and re- 
lated infrastructures and activities for implementing standards through 
programs of certification, metrology, and quality control have been of 
great importance to many countries. And as more countries undertake 
programs of industrialization, these subjects take on even greater in- 
terest in the future. Futhermore, UNIDO has collorborated with the In- 
ternational Standards Organization (ISO) in identifying the importance 
of standardization and related disciplines in the transfer of technol- 
ogy' 1 including the adoption and/or adaptation of international 

and other national standards in programs of transfer of appropriate 
technology. 

Over the past several years UNIDO has provided the assistance of ex- 
perts in standardization and related disciplines to no less than thirty 
countries. In most instances, these experts fprm the advisory component 
of UNIDO projects with the respective countries. These projects often 
include additional assistance such as buildings, equipment, and inter- 
national training fellowships for local staff. Some projects run as 
little as three to six months. Others provide continuing assistance 
well into phases of refinement of the national program. An example of 
the latter is the program of UNIDO technical assistance to the Thai In- 
dustrial Standards Institute, which is now in its eigth year apd with 
which the author was associated in its early development/--*'*' '* 

THE UNIDO/NSO PROJECT 

The UNIDO project with the Nigerian Standards Organization (NSO) of 
the Federal Ministry of Industries, Nigeria, was approved on October 1?, 
1975. The project has a long range objective "to promote the industrial 
and economic development of Nigeria through the development of standard- 
ization, quality control, industrial metrology, and metrication acti- 
vities and their application in the industrial and business operations 
carried out in the country. It is expected that this would contribute 
towards raising the quality of industrial goods manufactured in the 
country, facilitating trade between Nigeria and its trading partners and 
bringing about an added value to Nigerian products sold abroad, thereby 
raising the nation's GNP" . 

Specific objectives resulting in major activities under the project 
include , 

1. Strengthening of NSO management and organization. 

2. Development of suitable buildings and facilities for an NSO cen- 
tral testing laboratory. 

3. Establishment of central and subsidiary laboratories for testing, 
quality control (certification), metrology and calibration. 

4. Strengthening of NSO's quality inspection programme and quality 
certification marking scheme. 

5. Training of NSO Staff. 

6. Training of industrial personnel. 

Advisers and consultants provided by the project are to include the 
following, 

1. Industrial quality inspection (certification). 

2. Industrial metr^l; ,.v, 
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3. Chemical Analysis. 

4. Materials testing. 

5. Programming and construction of buildings and facilities for 
quality control and testing centres. 

6. Laboratory testing-management , operation and selection of labor- 
atory equipment. 

?. Development and conduct of training-consulting workshops on 

quality control principles and techniques including quality as- 
surance programmes and certification marking schemes. 

The project became operational with the author's assignment as team 
leader and adviser on industrial quality inspection (certification) in 
January, 1977 for the 1977 calendar year. During the year an additional 
consultant on programming and construction of buildings and facilities 
completed an initial effort in the development of a permanent office and 
laboratory facility for NSO, Additional project activities are program- 
med through 1981 for implementation. 

NIGERIA 

Nigeria is located on the west coast of the African Continent. It 
has a land area of approximately 913*000 square kilometers. It is en- 
dowed with vast national, mineral and human resources, including oil and 
a population of over 9° million. Thus, it has one of the greatest po- 
tentials to develop an economy, culture, and industry resulting in a 
high quality of living for its citizens. 

Nigeria is intensifying its efforts for a rapid industrial and eco- 
nomic development of the country. The Third Five-Year Development Plan 
(1975-1980) outlines the development strategy in the various sectors of 
the economy including emphasis accorded to the development of the in- 
dustrial sector. This includes promotion of local industries for the 
production of commodities and services required both for the local mar- 
ket as well as for export purposes. An additional strategy is the de- 
centralization of industries and related support services. This is 
being effected through the increasing involvement of state governments in 
planning industrial activities within their respective states as well as 
the location of Federal Government financed industrial projects at suit- 
able rural areas. Nigeria's financial capacity has been increased con- 
siderably by the current world economic situation, especially with res- 
pect to the oil industry. This has, in turn, given rise to a burst of 
intensive industrial activities particularly in the construction sector 
and related areas. 

In view of the above, it has become necessary to set up and/or 
strengthen the institutional and technical services infrastructure in 
the country. One of these is the Nigerian Standards Organization, 

THE NIGERIAN. STANDARDS ORGANIZATION 

NSO was officially established by the Nigerian Standards Organiza- 
tion Decree 1971* Decree No 56, which was promulgated in December, 1971 
with a commencement date of 1 January, 1970 when the Organization act- 
ually started to function. The decree lists the principal functions of 
the Organization as follows, (5) 

"1. to organize tests and do everything necessary to ensure compli- 
ance with standards designated and approved by the Council, 

2. to undertake investigations as necessax'y into the quality of 
facilities, materials and products in Nigeria, and establish a 
quality assurance system including certification of factories, 
products and laboratories, 

3. to ensure reference standards for calibration and verification of 
measuring instruments, 

4. to complie an inventory of products requiring standardization, 

5. to compile Nigerian standards specifications, 

6. to foster interest in the recommendation and maintenance of ac- 
ceptable standards by industry and the general public, 

7. to develop methods for testing of materials, supplies and equip- 
ment including items purchased for use of departments of the Gov- 



ernment of the Federation or a State and private establishments, 

8. to register and regulate standards marks and specifications* 

9. to undertake preparation and distribution of standards samples, 

10. to establish and maintain such number of laboratories or other 
institutions as may be necessary for the performance of its func- 
tions under this Decree, 

11. to compile and publish general scientific or other data, 

12. to advise departments of the Government of the Federation or a 
State on specific problems relative to standard specifications,, 

13. to sponsor such national and international conferences as it may 
consider appropriate, 

14. to coordinate all activities relative to its function throughout 
Nigeria and to cooperate with corresponding national or interna- 
tional organizations in such fields of activity as it considers 
necessary with a view to securing uniformity in standards spec- 
ifications, and 

15 . to undertake any other activity likely to assist in the perfor- 
mance of the functions imposed on it under this Decree.” 

For carrying out these functions, NSO at present has a staff of 
thirty-five professional personnel including a Director and an addition- 
al support staff of approximately fifty. This is presently below au- 
thorization levels and will need to be increased by additional recruit- 
ing. The Decree also established a governing body of the Organization 
known as the Nigerian Standards Council. The Council consists of some 
fourteen members representing a broad spectrum of government, industrial, 
and other organizations , such as Agriculture, Transport, Health, Works, 
Mines and Power, Industry, Science and Technology, and Universities, a 
consumers' council,, and private industrial firms. The principal func- 
tions of the Council under the Decree are, 

"1. . to advise the Federal Military Government generally on the na- 
tional policy on standards, standards specifications, quality 
control and metrology, 

2. to designate, establish and approve standards in respect of metro- 
logy, materials, commodities, structures and processes for the 
certification of products in commerce and industry throughout 
Nigeria , 

3. to provide the necessary measures for quality control of raw 
materials and products in conformity with the standard specifi- 
cations, 

4. to determine the overall policy of the Organization, in particu- 
lar with regard to the financial, operational and administrative 
programmes of the Organization and to ensure the implementation 
of the said policy, and 

5. to carry out other functions imposed on it under this Decree or 

any other written law.” 

To date, NSO has published just over 80 national standards and codes 
of practice. More than that number of additional standards are now in 
various stages of committee work, approval and publication. They cover 
a broad range of consumer and industrial products as well as test methods 
in textiles, soaps, safety matches, candles, batteries, plastic bowls and 
buckets, milk, ball point pens, enamelware, white bread, glass bottles* 
cement, building blocks and bricks, steel wire and reinforcing bars, 
fans, crash helmets, auto and bicycle tires, electric bulbs, electric 
fuses, paper, gas cylinders and other subjects of economic importance. 

The Organization has patterned itself on the principle of consensus 
of industrial, commercial and consumer opinion adopted by most standards 
organizations . The standards are drawn up by committees of specialists, 
the members of which are drawn from manufacturers,, major users, govern- 
ment departments, universities, and any other organization with a close 
professional and business interest in a particular product. NSO itself 
provides the secretariat for these committees and is staffed with qua- 
lified men and women who prepare drafts, initiate any research work need- 
ed, survey factories, evaluate tests, and generally service the committee 
operations. Approvals for standards promulgation are through the Nig- 
erian Standards Council and the Federal Ministry of Industries (FMIND). 



Internationally, NSO is a full member of ISO representing Nigeria 
with that international body and provides documentary information in 
Nigeria on all ISO member bodies including foreign national standards and 
international standards. It is also the subscriber member of the Inter- 
national Electrotechnical Commission (IEC) and is Nigeria's contact point 
for the Codex Alimentarius Commission, the United Nations Joint FAO/WHC 
Food Standards Programmes. 

Regionally, NSO is a founding member of the African Regional Stand- 
ards Organization (ARSO ) . NSO was also elected to the first three year 
term of presidency of ARSO at the founding conference, January 10-17, 
1977. 


Implementation of standards was generally on a voluntary basis until 
1976 when an amendment, Nigerian Standards Organization (Amendment) Deo- 
ree 1976, Decree No. 20, further empowered the Commissioner (FMIND) tb 
declare, by order published in the Government Gazette, any standard es- 
tablished under the principal Decree to be a Mandatory Industrial Stand- 
ard, making it an offence for any manufacturer (and importer) to fail tc 
comply with any of the requirements of such a standard. The Commission- 
er used the occasion of the opening of the NS0/UNID0 Training-Consulting 
Workshop on Quality Control Principles and Techniques,, University of 
Lagos, on 5th September 1977 to announce mandatory standards on milk and 
cement . 

The NSO Decree (Section 10) authorizes the Nigerian Standards Coun- 
cil to permit the manufacturer of a product which has been found to con- 
form to a relevent Nigerian Industrial Standard to affix the NIS Cer- 
tification Mark to a conspicuous part of its products or packaging. 

This program was first implemented by issuance of license to use the NIS 
Mark on toilet and laundry soaps to a major manufacturer on October 
197^ (World Standards Day). On the occasion of the following year's 
World Standards Day, 14 October 1975»- licenses to use the NIS Mark were 
issued to seven additional companies, representing textiles, candles, 
enamelware safety matches and plastic products. Other companies are 
under study for licenses to use the Mark, but no additional licenses 
have been issued to date. This activity is expected to be enhanced 
considerably by the establishment of a functional department on Gertifi* 
cation and Quality Control (C & QC) in October, 1977* 

CERTIFICATION AND QUALITY CONTROL 

NSO's certification program is expected to grow rapidly and serve 
a vital link in the implementation of standards,, including raising of 
quality of locally produced and imported commodities and products. 

The first phase of the UNIDO project with NSO served to identify and 
provide training in areas of further development needed to strengthen 
the certification program. The establishment of a functional department 
is a significant initial step. Other steps envisaged for the program 
and certification group are v6) ( 

1. Staffing and manpower planning for the C & QC department. It is 
expected that some personnel may continue as secretaries to a 
limited number of technical committees in the preparation! of 
standards with gradual relinquishing of those duties. Addition* 
ally, the certification group will continue to call upon specific 
expertise from other NSO staff (e.g. textile technology, electri- 
cal technology, etc.) for its work with certain companies when 
such expertise is not present among those assigned to the group. 

2. Organization of the information and working files associated with 
the present eight licensees. Updating of the factory visits, 
tests, etc. associated with these licensees. Uniform programs Of 
product marking, promotion and advertisement of the NIS Mark for 
consumer education and encouragement of other manufacturers. 

3. Development of rules and regulations pertaining to the operation, 
implementation and maintenance of the certification program with 
neat, attractive and accurate brochures for distribution to po*> 
tential licensees. Some specific items include,. 

a. A General-Certification Mark Licensing Procedure, 

b. Regulation pertaining to application and issuance of licensee 
to use the quality mark including the official application 


form, and the official license form for voluntary and mandatory 
— domestic and import certification, 

c, Regulation giving specifications of the standard mark,, 

d. Regulations giving rate of fees, etc. 

4. Development of Schemes for Surveillance Inspection and Testing 
(as given in Appendix A of reference 3). 

5. Planning the expansion of the certification program. Some of the 
directions envisaged are, 

a. A listing and survey of companies producing products for which 
there is a Nigerian standard. Follow-up mailings and visits 
to advise, assist and encourage these companies, 

b. Implementation of mandatory standards by the certification 
program ,. 

c. Consideration of certification to other than national stand- 
ards, 

d. Location and accreditation of testing laboratories, 

e. Development of an export inspection program. 

6. Continued activation of the technical committee on Quality and 
Reliability including, 

a. Establishment of a National Association of Quality Control, 

b. Preparation of standards on quality control principles and 
techniques such as sampling tables, control charts, quality 
assurance programs, etc. 

?. Establishment of a technical (working) committee on testing and 
calibration to plan tests and coordinate testing facilities and 
programs in support of the certification program. 

8. Pursuing of international training opportunities for C & QC staff 
via UNIDO fellowships or other programs. 

9. Continued coordination of the Training-Consulting Workshops on 
Quality Control Principles and Techniques for industrial person- 
nel . 

CERTIFICATION AND MANDATORY STANDARDS 

With the issuance in 1976 of Decree 20, NSO (Amendment),, NSO is 
empowered to enforce mandatory standards. It has been recommended that 
this activity be accomplished via mandatory certification on these 
standards (5b. above). Some additional remarks pertinent to this mat- 
ter follow. 

Whatever the means of enforcing,, to be effective, mandatory stand- 
ards require adequate staffing and a program of surveillance inspection. 
Sometimes, the tendency to declare standards as mandatory becomes very 
strong in the thinking that this is the only sure way to make progress 
in standardization . Often the opposite effect is achieved. Especially 
if the surveillance and enforcement cannot keep up with the declaration 
of mandatory standards. Planning for the certification group must be 
coordinated with the effort to establish mandatory standards. One so- 
lution of the above problem may lie in a better unders tanding of various 
forms of compulsion. There are at least two levels of compulsion which 
may be created. 

One is referred to as a "subtle compulsion", obtained by the politi- 
co-technical activity of convincing responsible authorities to specify 
or require quality in products and services. Large Government and com- 
mercial organizations, purchasing supplies and services from local busi- 
nesses, can create a compulsion by setting required levels of quality 
via quality requirements in purchase contracts, building specifications, 
etc. and by giving preference to those businesses having certified pro- 
ducts under the voluntary program, or even requiring such certification. 
This has the feature of providing an option to a local producer — he can 
opt to direct the manufacture of his product toward this business or he 
can decide not to compete in that arena and seek other markets. More 
promotion of this type of compulsion may accomplish equal benefits and 
be easier to manage. 

The second form of compulsion is referred to as "hard core compul- 
sion" obtained by the politico-technical-legal activity of decreeing by 
law that imported and/or manufactured products must conform to a given 
standard — with associated liabilities. Mandatory standards with re- 



lated certification is of this type and has its place in the overall pro- 
gram of industrial development. Some quality requirements need enforce- 
ment from an organization or program with a broader perspective than the 
manufacturer himself or his immediate customer. For example, the higher 
cost of electricity or even the cost of a related fire from poor quality 
electrical cable is most often not borne by the manufacturer of the 
cable or by the construction contractor — without some independent 
enforcement of such quality, they may be tempted to cut their in- 
dividual costs by supplying a low grade cable. 

Generally, products involving personal injury or economic loss are 
subject to mandatory standardization and certification. The declaration 
of mandatory standards should be used as sparingly as possible, and then 
within the capability of implementation. 

The procedures by which mandatory standards are declared should be 
examined in Nigeria. Consideration should be given to a formal pro- 
gram of 

( 1 ) a public announcement of intent to establish a mandatory stand- 
ard. 

(2) an appeal procedure and time interval to receive and hear com- 
ments from those affected, perhaps including a public hearing, 

(3) public notification of final decision with a prescribed grace 
period for start of enforcement. 

TRAINING PROGRAMS 

Under the UNIDO/NSO project, at least four types of training pro- 
grams are considered and have begun. These are, 

1 . UNIDO Sponsored International Training Fellowships for NSO staff. 
And these are of two types, (a) group training programs on gen- 
eral subjects of standardi zation , quality control and related 
topics and (b) individually tailored programs on specific sub- 
jects of interest and need to the Organization. Programs of the 
first type include courses in Japan, India, Australia, Sweden, 
U.K., USSR and other countries on a rotating regional basis. 

NSO staff has participated in all of these. Programs of the 
second type take place mainly under the project and require in- 
dividual coordination. Several are under development. 

2. In-country Training Programs for NSO Staff. These programs are 
generally conducted by UNIDO Advisers assigned to the project. 

An extensive program was carried out by the author during the 
first half of 19?? on topics of Quality Control Principles and 
Techniques, Standardization and Certification. Other programs on 
Metrology, Laboratory Testing, Chemical Analysis, etc, are en- 
visaged. 

3. Seminars, Conferences, and Short Term Workshops for NSO Staff and 
Other Organization Personnel. These programs are organized 
through various associations and organizations to disseminate 
information and promote programs of potential benefit. Related 
programs during 197 ? were, 

a. Seminar on, "Evolutionary Operation: A Method for Improving 
the Performance of Existing Industrial Processes." to the 
National Chemical Engineering Society, Engineering Lecture 
Theatre, University of Lagos, March 26, 1977 • 

b. One-Day Workshop for Quality Control Managers of the Techni- 
cal Committee on Quality and Reliability on "Quality Control 
Techniques, Quality Systems, National Associations for Quality 
Control and National Standards on Quality Methodology", NSO 
Conference Room, April 22, 1977* 

c. Seminar on "Development of National Standardization and Certi- 
fication in Thailand and its Implications for Nigeria", Con- 
ference Room, Federal Institute of Industrial Research (FUR), 
Oshodi (Lagos), May 13, 19 77. 

d. Four-Day Workshop by UNIDO and FUR for industrial personnel 
on "Quality Control in the Textile and Allied Industries", 
Conference Room, FUR, Oshodi (Lagos),. June 20-23, 1977* 



Conducted by two UNIDO Advisers, FUR staff and speaker from 
Kaduna Polytechnic University. 

4. Training-Consulting Workshops on Quality Control Principles and 
Techniques for Cost Reduction, Process and Product Improvement 
and Control for industrial personnel with NSO staff observing and 
assisting. These programs were conducted using the "sandwich” 
principle of several days of introduction lectures in a classroom/ 
conference setting, a period of in-factory consultation during 
which each participant returned to his own factory to work on a 
problem and/or application with consulting visits by the workshop 
leader accompanied by NSO staff, and concluding lectures and dis- 
cussions back in the conference' room to summarize findings, ex- 
plore additional techniques and promote future applications. 

Three workshops were conducted as follows, 

a. Lagos Area, University of Lagos Continuing Education Centre, 
September 5-23, 19??. Thirty-seven participants representing 
twenty-seven companies attended this workshop. Products re- 
presented include, pharmaceuticals, cosmetics, beverages, 
foods, confectionery , doors, electrical fixtures, enamelware , 
paperboard, air conditioners, wire, truck assembly, Venetian 
blinds, paints, metal cans, etc. The workshop was opened by 
the Federal Commissioner for Industries drawing national at- 
tention to the program. The Manufacturers Association of 
Nigeria (MAN) assisted NSO in the circulation of the workshop 
announcement . Consultations in every factory highlighted areas 
of applications for quality control techniques and quality sys- 
tems principles and several significant cost reduction efforts 
were realized within the workshop period. 

b. Kano Area, Auditorium, College of Preliminary Studies and 
Conference Room of the Industrial Training Fund,. Kano, October 
17-28, 19??* Seventeen participants representing fourteen 
companies attended this workshop in addition to seven staff 
members of NSO, Products represented include, printing, shoes, 
textiles, cosmetics, pharmaceuticals, bicycles, steel con- 
struction, furniture, baked goods, cartons, etc. The Kano 
workshop was opened by the Kano State Commissioner for Indus- 
try, Trade and Cooperatives and also uncovered quality control 
applications and significant waste and cost reductions. 

c. Kaduna Area, Murtala Mohammed Square Conference Centre, Kaduna, 
November 7-18, 19??* fwenty-nine participants representing 
ten companies attended. Products included, textiles, defence 
products, printing, newspaper, yeast and alcohol, sugar, live- 
stock feed, auto assembly, etc. As with the others, signifi- 
cant applications were uncovered. 
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series of workshops are the beginning of a vital program of 
assistance to industry by NSO and are seen to accomplish the 
six benefits, 

provide direct training for the participating industrial per- 
sonnel, 

expose participating NSO staff to more industries and consult- 
ing-investigating experience, 

provide much needed publicity for NSO in industry as well as 
in its own government organizations, 

provide a mechanism for encouraging industry cooperation with 
NSO's certification program, 

provide direct process and product improvement applications 
within participating industries, and 

initiate the quality improvement spiral of better products at 
lower costs and prompt delivery for Nigerians. 


Continuation of these workshops has been programmed in the up-dated 
project document and activities and coordination by the C & QC depart- 
ment is recommended above. 


CONCLUSIONS 

Countries the world over have implemented and are continuing to 
implement national programs of standardization, certification, and qua- 
lity control resulting in tremendous national economic progress. As a 



specific example, the Japanese have converted their reputation of cheap, 
shoddy products to a reputation for high quality, precision products at 
world competitive prices. 

It was accomplished by a multi-faceted effort throughout their so- 
ciety, including export inspection laws and implementation; standardiza- 
tion and certification laws and implementation; widespread education 
through radio and press, in-plant and outside plant training courses, 
management seminars, industrial team visits abroad; employee motivation 
for quality including sharing of profits to raise the standard of living 
of their employees who become company customers; invitations to experts 
from developed countries, including heeding their advice and taking ac- 
tion to implement corrective efforts; to name some significant efforts 
in their awe inspiring climb to a world leader of quality products. 

It can happen in Nigeria. It will take the kind of effort and co- 
operative spirit mentioned above to bring it about. What more appro- 
priate time to launch such an effort than the post-era of FESTAC-II 
with its inspiration to unity of purpose and spirit of accomplishment 
and influence on a competitive world! 
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CAQ STARTS WITH CAD 


William Riemenschneider 
Quality Information Equipment Engineer 
Aerospace Electronic Systems Department 
General Electric Company 
Utica, New York 13503 


INTRODUCTION 

Total Quality Control requires that Quality be designed into a product. As 
computer resources are developed for use in the design activity, the quality function 
must become directly involved in incorporating its requirements into the design and 
manufacturing processes generated by Computer Aided Design (CAD) and Computer Aided 
Manufacturing (CAM) . Computer Aided Quality (CAQ) techniques should be incorporated 
simultaneously with the design of the CAD/CAM data base. 

This paper discusses methods for superimposing control techniques on the CAD 
System while developing thermal analysis, reliability prediction, maintaining 
configuration control and developing test and inspection programs. 


DESCRIPTION 

Conversion of traditional design, manufacturing, and quality methods to Computer 
Aided methods is generally done on an individualized task basis to solve a specific 
problem. This results in a series of stand-alone programs which require either 
manual interfaces or special interface programs. Automating on such a basis can 
result in a very efficient single-step operation that solves the specific problem, 
but in many cases, complicates the overall interface problem with other functions in 
the design-to-shi.p cycle. Figure 1 illustrates the conventional information flow 
based on manual development of design data. 
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Figure 1 : Conventional Information Flow Diagram 
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Many steps are never automated for several reasons: 

Conversion is not considered to be cost effective. 

Traditional operating rules and responsibilities are rigidly applied. 

By developing an interactive and functionally integrated automated system, in 
module form, a real gain in accuracy, productivity, efficiency, accountability, and 
flexibility can be achieved. Such a system should be based on the following 
premises : 


Everything designed will be manufactured. 

Everything manufactured will be quality evaluated. 

Manufacturing processes will be quality evaluated. 

Quality is designed into a product. 

Every step can be automated. 

Figure 2 illustrates the modularized interactive Computer Aided System concept. 
It differs from traditional systems in that it is developed as an automated system 
that permits manual operations. It is not a manual system with automated steps. It 
also differs from some CAD/ CAM type systems in that its modularity permits each 
function to retain total control of their areas of responsibility. 

The links between modules are designed for automatic communications (hard wire) 
but can be used with paper tape, magnetic tape, punched cards, etc., as the transfer 
media. A mandatory requirement is that the I/O format and transmission code be 
compatible with the communications line requirements. The selection of the 
communications method is the responsibility of the user of the data. 

The key element of the system is the centralized Data Base. This data base is 
defined by representatives from all functions but is operationally controlled by an 
independent group. Data contained here is generic in nature with all redundacies 
removed. Data may be implicit since a computer talking to another machine does not 
require explicit information. This permits a reduction in paperwork but requires 



FIGURE 2: Interactive Automated System 












that strict revision/configuration control measures be incorporated. 

Modularity permits elements to be developed by the responsible organization to 
any size, in any configuration, and at any time in a logical and controlled sequence. 
Modules can be bypassed, omitted, or be manual operated without affecting other 
operations. Operation of a module is controlled by the user thereby providing him 
with direct cost and efficiency control. Modularity also permits functions to 
interact in an automated mode by providing timely and accurate feedback. Audit 
points, information checks, and production "stop" points can be programmed into 
various modules and stored on the central data base. 

Use of the system is illustrated by the two examples shown in Figures 3 and 4. 
Figure 3 depicts the electrical design through electrical test of a digital printed 
circuit board. This flow is presently in use at GE-AESD. Figure 4 describes the 
design through inspection of metal parts. This flow is only partially operational. 

In both examples, each data base is physically in the same location which is trans- 
parent to the user. Strict data base access and content change controls exist. 



PCB Flow Manufacturing 
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The digital circuit flow description 
begins where Engineering selects the elec- 
trical components to be used. A complete 
description of each component is entered 
onto a computer file (CAD #1) . A Quality 
check is made, via an automatic printout 
(CAQ #1) which is used to verify either the 
existence of these components on the 
Department's qualified component list or 
that a request for qualification has been 
processed. If descrepancies exist, appro- 
priate corrective actions are initiated 
immediately . 

Component placement programs are run 
(CAD #2) with the result placed on the data 
base. This data is then passed through a 
thermal analysis program and then through 
a reliability analysis program (CAQ #2) . 
Reliability Engineering reviews the output 
of CAQ #2 and must approve the result or 
further development is not permitted. Con- 
currently, a component approval report is 
again generated (CAQ it 3) comparing the 
components listed in the placement program 
with those on the department's approved 
parts list. This is done to assure that no 
unauthorized components have found their 
way to the design. 

The circuit board interconnections 
are designed via another computer program, 
CAD it 3. Drafting extracts the data via an 
automated plotter to produce artwork and 
machining master photo plots. The contact 
positives made from the master photo plot 
are checked and approved by Design Engi- 
neering. Only when this approval exists 
can Manufacturing or the Test and Inspec- 
tion function extract the data. 

The manufacturing extract (CAM it 1) is 
actually three separate activities. 

1. Develops negatives used to develop 
the glass plates used in the 
printed circuits board (PCB) etch- 
ing process. 

2. Develop an NC program to drill the 
PCB. 

3. Develops an NC program to perform 
the router operation on the PCB. 
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Figure 3: Digital PCB Flow (continued) 
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Quality Information Equipment Engi- 
neering (QIEE) extracts data via programs 
which design the test adapter and provide 
an NC program to test the continuity of the 
resulting PCB (CAQ #4) . 

Concurrent with the CAM 1 and CAQ 4 
activity, Design Engineering develops the 
final performance criteria by utilizing 
Fault Analysis Simulation programs (CAD #4) 
to measure design effectiveness and estab- 
lish test requirements. QIEE software 
(CAQ #5) utilizes the Engineering output 
to develop a test program for use on 
computer controlled test systems. To 
ensure thorough testing, the CAD 4 and 
CAQ 5 operations generally require several 
iterations early in a new product design 
before acceptable results are obtained. A 
significant advantage is the reaction time 
and the ability to measure the effective- 
ness of the test to determine all potential 
faults in the product. 

Traceability of all data is maintained 
throughout by use of revision letters, 
correction counters, and database change 
time and date records. 

Machine parts are designed (Ref. Fig. 4) 
using an Interactive Graphics Terminal 
(IGT) . When complete, the design is stored 
on the product data base which is used by 
peripheral equipments such as plotters and 
printers to provide Engineering drawings, 
parts list, and other required design infor- 
mation. This data base is also used by 
Manufacturing, via an IGT to add informa- 
tion required to develop programs for use 
by Numerically Controlled (NC) or Computer 
Controlled (CC) machines. This additional 
data is stored on a separate section of 
the product data base where it and the 
Engineering data is accessible by the 
Quality function, using an IGT, for use in 
developing the program to be used by the 
automatic inspection equipment. Manual in- 
spections are possible in instances where 
a terminal is located at an inspection 
station. This terminal would display the 
requirements and act as an entry point to 
feedback results. The Quality requirements 
are stored on a separate section of the 
product data base. 

Inspection results are collected, 
either automatically or manually, and trans- 
mitted to a separate section on the product 
data base using an interactive data termin- 
al (IDT) . Process and product analysis 
reports are developed which provide feed- 
back instructions to the machine tools. 

This feedback is presently manual but auto- 
matic operations are in the discussion 
phase . 


Figure 4: Machined Parts Flow 











SUMMARY 


The integration of CAQ is not limited to any type of design. It can be, indeed 
it must be, included into the total design and manufacturing operations. It should 
include test and inspection results reported automatically. 

The operating benefits of a modularized total system are many and varied. They 
can be realized only if existing methods, standards, instructions, and organizational 
responsibilities are considered as starting guidelines and not end goals . The follow- 
ing is a list of some of the advantages identified in the use of such a system at 
GE-AESD. 


o Provide a common data base for the storage and extraction of data 
o Provide an effective control of data 
o Minimize redundancy of data 

o Reduce schedules (Lead Time) 

o Minimize liaison between all concerned functional groups 
o Consistent interpretation of data 

o Consistency of output (formats) 

o Minimize recurring manufacturing and quality programming 
o Provide back-up operating programs 

o Manufacture to design and standards 

o Quality measurements to design and standards 
o Configuration control 

o Revision control 

o Historical records 

o Quality information reporting - consistent and timely 
o Error checking (designed within manufacture and quality capability) 
o Improved cost assignment and visibility 
o Defect visibility and responsibility assignment 
o Minimize print development and control problems 

o Output extract independent from input loading 
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ORCHESTRATION - THE TRUE TEST OF QUALITY MANAGEMENT 


Paul J. Yamilkoski 
Manager Process Compliance 
Avionics & Missiles Group 
Rockwell International 
Cedar Rapids, Iowa 52406 


INTRODUCTION 

To put my discussion of today's topic, which is "Orchestration - The True Test 
of a Quality Manager", into perspective, I feel that some introductory remarks about 
my organization are in order. 

The Collins Divisions of the Avionics and Missiles Group of Rockwell Internation- 
al are an over $200 million dollar organization consisting of three independent 
Profit Centers addressing separate aviation markets. Each of these Profit Centers 
has their own Quality Control function and are solely responsible for the quality of 
their manufactured products. Each of these Profit Centers is also solely responsible 
for their marketing, scheduling, profit and loss. For this reason each of these 
Profit Centers have organic Marketing, Engineering and Manufacturing activities and 
are, in turn, supported by the common support functions of Purchasing, Finance, Per- 
sonnel, and Product Assurance. 

The Quality Control Managers have dotted line relationships to the Product 
Assurance Director. The Product Assurance Director is responsible for Procurement 
Quality, Product Evaluation, Receiving Inspection, Component Engineering, Reliability, 
Product Integrity and Product Assurance Engineering. He is responsible for the over- 
all Quality System and establishes Quality Policies and Standards. He tracks Quality 
Cost in the four major classical categories of Preventive, Appraisal, Internal and 
External Failure Costs. 

Having established this common data base of information, I would like to address, 
in the next 25 minutes; the significance of Orchestration; some prerequisites for in- 
suring that it takes place; what should be done to achieve orchestration; what we do 
within the Collins Divisions to achieve it; the role computer aided techniques play in 
this effort; and what lies ahead to be done in the future. 

ORCHESTRATION - WHAT IS IT 

Juran addresses this subject, and, in general terms, states that Orchestration is 
the coordination, performed by the Quality Manager, of the entire collection of acti- 
vities through which we achieve "fitness for use", no matter where these activities 
are performed! 1 ) In our organization this Quality Manager is the Product Assurance 
Director. 

Orchestration includes the coordination of the Quality aspects of: 

. Receiving activities 

. Materials handling, storage and issue activities 

. Special process activities for fabricating, finishing and assembling 
our manufactured product 

. The basic in process assembly, inspection and test activities 

. Final inspection and material handling activities 

. Packaging and shipping activities, and 

. Customer services on returned goods 
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to the extent that internal quality indicators in each of these areas monitor and 
signal problems and to the extent that these indicators track with our customers 
opinion of our products "fitness for use". In general terms, "fitness for use" is 
another way of saying the product is "OK". But, as applied in our customer interface, 
it also includes understanding how the product is used and how that usage relates to 
non-verified rejections. 

PREREQUISITES THAT PERMIT ORCHESTRATION 

The prime prerequisite to the ability to even possibly perform this task is a 
"commitment by top management to the quick detection of quality troubles at the. 
earliest possible point in the process after defectives have been generated".^) Why? 
Not just because intuition tells us that rework/repair has a degrading effect on the 
quality of our products, but, in terms most popular and meaningful to management, it 
is expensive and results in a direct reduction of potential profits. 

ORCHESTRATION - THE IDEAL 

For the sake of this discussion let us consider what can be considered to 
approach an ideal . 

The Ideal (Figure 1) would be to have a fully automated system that provided, as 
product flowed to the customer, feedback and positive corrective action on identified 
discrepancies at the key automated inspection and test points throughout the pro- 
cesses with the resultant plot of defects being attributable to truly random part 
failures. I mentioned earlier that the commitment by management should be there, not 
only because early detection of defects results in better quality, but early detec- 
tion of defects results in a cheaper defect, a less costly rework or repair. As an 
example, it has been our experience that an "electrically defective part" found with- 
in the receiving inspection activity saves a $12.00 rework cost that would have been 
expended if detection occurred during assembly or test (not counting the Part Material 
Costs). We have also observed that a detected assembly inspection workmanship defect 
cost $5.00 to repair; test rejections, during module test, costs about $12.00 to re- 
pair; final radio rejections cost $10-20/repair (unless it occurs during burn-in of 
the end item at which time it would cost $27,00 to repair); and, last but not least, 
customer warranty rejections range from $60-$250 to repair depending on equipment 
complexity and defect. As one can see, there is substantial financial motivation to 
reach for the ideal . 


WHAT CAN BE DONE? 

Needless to say, we within Collins Avionics Divisions have not, in our opera- 
tion within Avionics and Missiles Group reached the ideal, however the closer we move 
towards that objective the more we have realized. 

. Our dependence on computer aided quality, and 

. Productivity gains which earn us an improved return on our investment. 

The true test of Manufacturing Management is the establishment of a functioning 
entity that balances the needs for long term profitabil ity, in terms of on going 
sales, with the needs of a quality product in the market place. 

The true test of the Quality Manager responsible for overall Quality Systems 
definition and policy is understanding what affects on going sales from a quality 
viewpoint, communicating these needs to General Management, and orchestrating the 
activities of the company that can contribute significantly. 

Let us look at these activities. It has become apparent to us that, as the 
business grows, as technology continues to become paradoxically simpler but yet far 
more complex, we are but one of many "meters" among the many manufacturing concerns 
that both stimulate that continued technological growth and, at the same time, 
assist in the harnessing of it. 

Let me expound on this briefly and then, having done so, tie together the ana- 
logies used so far in this discussion. 
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FIGURE 1 


Manufacturing Businesses, through recognition of the market place needs, initiate 
R&D activities that capitalize on new uses of existing hardware, and are the catalysis 
for new technologies. We either design and develop these technologies and resultant 
products, or we subcontract the effort as required. Regardless of the choice, we re- 
cognize that it is not until after multiple attempts, or many trial and error type 
situations, that the product becomes usable or economically feasible. 

Through Receiving Inspection test and inspection capabilities, through controlled 
process inspection and test, through machine controlled assembly, through the analysis 
of observed anomalies in each of these areas we identify problems and seek implementa- 
tion of corrective actions. We meter the processes and act to control. 

HOW DO WE DO IT? 

We do this through techniques that are also the outgrowth of technological ad- 
vances. Computer aided or computer controlled assembly, fabrication , inspection., and 
test machines and equipment supply us the data we need to continually assess and im- 
prove. 

If we had not tested every IC utilized within our equipments, we would not, nor 
would the industry in general, have seen the rejection rate of received products drop 
from four percent to half of one percent. This is why today we test all diodes and 
transistors. Not because product received is intentionally unsatisfactory, but be- 
cause all defects and defectives have not been eliminated and elimination of defec- 
tive items within receiving is much cheaper, as noted earlier in our discussion, than 
the elimination at an assembly or end item level. 

But receiving data is only part of the picture. Through semi -automated assembly, 
errors are reduced. Through automated inspection and test undetected errors, not 
caught in Receiving Inspection or earlier inspections and tests, are uncovered. What 
percentage distribution of these problems should be caught at Receiving? How many re- 
jections are design related? What problems observed in the field are vendor or manu- 
facturer design oriented? 

Orchestration assures answers to these questions and actions to accommodate 
corrections which result in happier customers and on going sales. 

OPPORTUNITIES FOR THE CONDUCTOR 

New products (i.e. CMOS, LSI, Bubble Memories) will require continued evaluation 
during their maturing process. With their advantages of reduced size and volume and 
increased capability, will come the basic problems of their complexity, when trying 
to determine the cause of rejection. 

As automated techniques take advantage of other technological developments in 
the form of "mini" and "micro" computer and processor capabilities, the tempo for 
defect information transfer will continue to escalate. 

Herein lies the opportunities as well as challenges for the conductor. The 
Quality Systems of the future must, to keep pace with the technology of the future, 
be responsive, dynamic and orchestrated. It must, of necessity 

. provide continued receiving adjustment 

. provide identification of undesired effects as soon as possible 
in the process flow 

. provide initial to final end product performance monitoring capa- 
bility 

. provide sensitivity to customer anxieties 

. provide professional vendor coordination relative to product deficiencies. 

Who is the conductor of this symphony of activities? With whom will these 
challenges lie? Who will be the object of this true test? You are! Thats who! 


(1) Juran, J. M. "Quality Control Handbook", Third Edition 1974, McGraw Hill Co., 
New York, NY pg 2-11 

(2) Schrock, Edward M. "Why Management Must Act to Improve Quality". Iron Age 
August 1977 pp 153-155 
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QUALITY: A CORNERSTONE FOR MANAGEMENT 


Wm. J. Blackburn, Quality Control Manager 
AMF Wheel Goods Division 
Olney, Illinois 62450 


Out of the business sector of western civilization a distinct 
social group has evolved with special interests, characteristic 
economic views, its own sense of logic, and an affinity for the tried 
and proven way of doing things. This group consists of those people 
who conduct, direct, guide, control, and have charge of a business 
enterprise. What has evolved is management. 

It should be noted here that the term management refers to a 
group of people; hence, it is a collective concept. Concisely, 
management is the collective power or leadership of a business. This 
leads to an intriguing question: Can management survive? To answer 
the question one must identify the determinants of management’s 
survival . 

In any business the survival of management depends on the 
continual economic success of the enterprise. The measure of economic 
success is the surplus of the enterprise without regard to the 
distribution of the surplus. In financial terms, surplus is the 
amount by which income exceeds expense. This surplus is often 

considered net income or profit. Income which generates a surplus 

is profitable income. It follows that the continual economic success 
of an enterprise requires profitable income. There are a variety of 
income sources. Some of them are: sales, revenue, private contri- 
butions, royalties, rents, leases, subscriptions, stock dividends, 
interest collections, and tax collections. In any case the success- 
ful enterprise must establish and maintain a composition of income 
which generates a surplus over all the expenses of doing business. 

If there is no surplus and the expenses of a business continue 
to exceed its income, the resources of the enterprise become 
exhausted, and there no longer exists the power to stay in business. 
Since management is the collective power of the enterprise, manage- 
ment's survival will be determined by its power to generate profit- 
able income. This also implies that if over a period of time expenses 
rise, management must demonstrate the ability to increase income. 

What does quality have to do with profitable income or the 
survival of management? This needs some clarification. Think of 
fitness for use. In purchasing an item or retaining a service, while 
comparing prices, you make your selection normally based on how fit: 
you think a particular choice will be for your use. What you are 
thinking about at that time is the quality of the items or services 
considered. Quality means fitness for use. When I refer to quality 
in what follows, I mean fitness for use. 

In order for management to survive it must be aware of that 
which produces profitable income. Management not only needs to be 
aware of it, it needs to establish and maintain it. What is it? 
Basically, income is derived through the trade of goods and services. 
The law of supply and demand determines the amount of income that 
can result from a particular trade. This is the crux of the matter. 
Demand is the amount of people who have both the desire for a 
commodity and the purchasing power to acquire it. The extent of this 
desire depends on the quality, which the consumer perceives. 
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From spring 1970 until the summer of 1976 the United States 
experienced one of the longest economic recessions in its history. 

It seems Americans were spared most of the indignities during this 
period that had been suffered during the Great Depression from fall 
1929 through spring 1935. However, Americans had a tremendous 
cushion in the early 1970's, which didn't exist in the 1930's. By 
1970 the United States had the largest financial reserve that any 
society has ever been able to amass: Thirty-four years of accumulated 
surplus funds in Social Security pensions, unemployment insurance, 
and welfare programs. On the eve of America's Bicentennial, Americans 
were becoming well aware that they had practically exhausted this 
reserve. 

Shortly after the recent recession got underway, in 1971 an anom- 
aly occurred: The beginning of the bicycle boom in the United States. 
It lasted until August 1974, and then the bicycle business followed 
the rest of the economy -- down! down! down! 

For over three years the demand for bicycles exceeded the supply 
by a considerable margin. During that period the average quality 
of a bicycle sold declined considerably. The customers wanted 
bicycles and weren't going to argue about color, a poor paint job, 
a sloppy looking weld, or even a missing part. I know the situation 
because I bought a bike during that period, which was before I got 
directly involved in that industry. I waited nine weeks for the bike 
I wanted, but settled for less when I was told that my order would 
be delayed indefinitely. I then looked for a bike in ten different 
stores . 

The best bike I could find not only displayed a few examples 
of shoddy workmanship, it came equipped with a 3-speed gear 
mechanism, that began malfunctioning within a week. The bike was 
purchased assembled and without any instructions or guarantee. So, 
when I took it back to the store, the department manager responded 
with, "The only thing I can do is replace it, but it will cost you 
another twenty dollars." 

It was a genuine, first class rip-off! I passed my opinion to 
the store's upper management and to that of the bicycle manufacturer. 
The manufacturer never responded, but the store did, granting me 
twenty dollars credit on my account. In turn, I have responded by 
still doing business with that store and resisting the use of any 
product of the bicycle manufacturer. 

When the bottom fell out of the bicycle market, it was not 
surprising to observe that business fell off rather proportionally 
throughout the industry. In the mind of the consumer there was 
little difference in the competitive commodities. Had one bicycle 
manufacturer consistently produced higher quality bicycles, it would 
have gained considerable market share, while its competitors' business 
declined more precipitously. There are fewer bicycle manufacturers 
today than there were back in the summer of '74. There simply was 
not enough quality in the first place to produce a profitable income 
for those companies that failed to survive. 

Normally, people don't plan to fail. But often they do fail to 
plan! In business, success usually comes through planning and that 

includes planning for quality. In the marketplace, if there is 
any competition at all, the commodity most suitable for use is the 
commodity in highest demand. 

There are many people with functional responsibilities and power 
over quality, who have been trying to sell management on the merits 
of quality. Most of these people are themselves part of management 
for some enterprise. This brings us to a paradox -- two of them in 
fact . 



First, management is regarded as the leadership of an enterprise. 
But the participants in the managerial ethic often find themselves 
so wrapped up in compromise, getting results, status and identifying 
with the establishment of the organization (which is usually bound 
in tradition) , that true leadership seldom surfaces. There is a 
reason for this. Leaders forge new direction, new ideas, new methods, 
and inventions. Managers on the other hand are inclined to control 
matters according to proven direction, proven ideas, proven methods, 
and proven products. If either of the two is going to come up with 

something better than before, it will be the leader because it 

wouldn’t be something better than before if it had already been proven. 
Those, who lead, martial the necessary forces and use their ability 
to actually test the new and better concept. The leader takes the 
risks. Good planning can minimize the risks and maximize the potential 
for a new concept. 

The second paradox is that often the quality professional is a 
part of management, but tends to mentally exclude himself from 
management by trying to sell management on quality. By this I mean, 
instead of taking charge and leading, as a part of management, this 
individual tends to consider himself separately and that his task 
is to sell those who have the power to establish and incorporate 
the philosophy and policies related to quality. This usually leads 
to frustration by the overwhelming sensation that the inertia of the 
organization is directed towards maintaining the status quo. Changes 
of philosophy and policy are best achieved through positive leadership. 

Positive leadership in the managerial arena requires an attitude 
of participation, authority, responsibility, good communication, 
attention to human behavior, and a willingness to venture for higher 
achievement of the enterprise. From the standpoint of management’s 
survival, there are several business organizations that already have 
adopted philosophies and policies which strongly enforce quality and 
the pursuit of excellence. Sometimes they have been developed by 
top management and trickled down, and in other cases they have surfaced 
through the courageous leadership of middle managers in the quality 
disciplines . 

It revealing to compare the performance of enterprises strongly 
oriented to quality vis-avis that of those which reflect little or 
no attention to quality. It is not my intention to name organizations 
that fit into either catagory. Research in this area reveals two 
phenomena. The first is that those corporations, schools, hospitals, 
clubs, and so on, which fail, seldom have any pronounced quality 
objectives or associates specifically responsible for obtaining 
quality objectives. The second is that those organizations which 
continually set strong objectives and are so well organized, as to 
routinely meet these objectives, out perform their competition in the 
long run. The latter are the enterprises which provide the goods 
and/or services most frequently in demand. This demand is the source 
of profitable income and provides the power for management to continue 
with the enterprise. 

Consumers around the world are becoming increasingly aware of 
quality and where there is a lack of it. The result is that in many 
democratic societies, where consumers feel that free enterprise is 
not properly attending to fitness for use, they are authorizing 
their governments to impose regulations that will mandate the attention. 
In other words, if management does not plan to take charge of quality, 
it appears the government may get the power to do so. In many cases 
where this has already happened the expenses of the enterprise sur- 
vived, while the enterprise did not - neither, of course, did its 
management . 

The place to start with quality is in considering the objectives 
of the enterprise and formulating clear statements regarding quality 
that: Will provide the basis for any related decisions and explain 



what is expected. In developing or revising quality policy the 
following factors should be considered:*^) 

1. The level of clientele in the market pursued. 

2. The position of leadership that the enterprise is to strive 
for versus competition. 

3. Consumer safety, product liability, etc. 

4. Availability, reliability, and maintainability of your 
product or service. 

5. Emphasis between conformance to specification and fitness 
for use, 

6. Product (or service) reliability and initial price 
relationship . 

7. Whether to optimize the costs of the user or producer. 

8. Reliance on controls by people or by systems. 

9. Line or staff responsibility for quality planning. 

10. Involvement of vendors and/or clients with the quality team. 

11. The role of top management in quality planning and 
assurance . 

12. Delegation of quality planning, assurance, and control 
authority . 

13. Emphasis on recognizing and reinforcing employee behavior 
which supports the quality policy of the organization. 

Management's statement of policy and adoption of philosophy - 
directly concerned with quality - is the starting point. This places 
quality attitudes in the desired perspective. It will serve as the 
mortar that will hold quality in the position that the enterprise 
needs for profitable income. This establishes quality as a corner- 
stone for management. 

As indicated, the agreement on quality philosophy and the related 
statements of policy are the starting point. Now the quality concept 
has to be put into action, which demands more leadership. This 
implementation requires planning, direction, organization, communica- 
tion, experimentation, inspection, and control. Yes, inspection and 
control. The reference here is not merely to the kind of inspection 
and control that is incumbent upon the individuals with the word 
"inspector" in their titles. Specifically, management must inspect 
performance to the quality objectives, direct changes when quality 
objectives are not being met, and reinforce behavior leading to 
the achievement of these objectives. 

This kind of leadership - concerned with quality - of an 
organization can result in higher degrees of consumer acceptance, 
lower selling expenses, lower operating expenses, and attention to 
breakthroughs - all of which enhance the potential for profitable 
income, the success of the enterprise and the survival of management. 



References 


1. For a skillful exposition relating the significance of quality 

to profit consult Juran, J. M. , The Quality - Profit Relationshi p, 
30th Annual Technical Conference Transactions, American Society 
for Quality Control. 

2. As a planning guide see Gryna, F. M. and Juran, J. M. , Quality 
Planning and Analysis , McGraw-Hill Book Co., New York, 1970 

3. For an adroit comparison on managers and leaders, Abraham 
Zaleznik's "Managers and Leaders: Are they different?" makes pro- 
vocative reading in Harvard Business Review , May-June 1977. 

4. For more specific considerations on quality policies and objectives 
refer to Juran, J. M. , Quality Control Handbook , Third edition, 
chapters 2 and 3, McGraw-Hill Book Co., New York, 1974. 


LCS 300:10:000 



QA AUDITING: A MANAGERIAL APPROACH 


Robert E. Del Ion, Assistant Director 
Management Services Group 
Washington Public Power Supply System 
Richland, Washington 99352 


INTRODUCTION 

It is now more than seven years since Appendix B to 10CFR50 was formally issued 
as federal law by the then AEC. Since this regulatory requirement was first advanced, 
company after company has scrambled to develop and implement QA auditing programs in 
response to Criterion 18 of Appendix B. 

Now that the nuclear industry has had significant years of experience with the QA 
auditing requirements of Appendix B, and experience with subsequent audit related ANSI 
standards, both draft and approved, I believe there is a need to put the managerial 
value of such efforts into perspective. Since there are sufficient standards, text- 
books, handbooks, training courses and procedures available on what a substantive QA 
audit program must comprise to be effective, I will not dwell on any of these areas 
other than to state that planning is pivotal to the effort. Logical, realistic, 
programmatic planning must be used throughout the entire process, whether it be in the 
formulation of an auditing philosophy, policy, procedure or plan, or in the budgeting, 
staffing, training, or implementation of the audit function. 

As you will quickly realize, I am a firm proponent of QA auditing. When executed 
with the strength of knowledge derived from training, experience, preplanning and 
professionalism, a comprehensive QA audit program can be an invaluable asset to a 
company. 


FUNDAMENTAL INGREDIENTS 

What are the fundamental ingredients of an effective QA audit program? As is true 
with any highly visible effort, the support of top management is basic - without it, 
for whatever reason, the program is rendered inadequate, a facade and a detriment to 
other QA activities. 

Recognizing the need for management support, how does one gain this valuable 
commodity? The answer is through the logical presentation of, and education to the 
applicable regulatory requirements and the benefits which can be derived therefrom. An 
important consideration in obtaining this necessary backing is the credibility of the 
individual or individuals advancing the QA auditing concept. The need for maintaining 
this credibility after program promotion and during program implementation is paramount. 
Loss of credibility will quickly translate into loss of management support and the 
start of a vicious downward spiral. Given top management support, the necessary fund- 
ing and staffing requirements can be developed and approved. 

The selection of personnel to staff QA auditor positions must be a rigorous, 
deliberate, well-thought-out process. The selection of capricious, argumentative, 
illogical, unprofessional or selfserving individuals can destroy an existing successful 
program, or delay for years the effective development of a burgeoning one. Auditors 
must foster understanding, maintain patience, be good public relations people, be 
capable of separating the chaff from the wheat, utilize others' time efficiently and 
judiciously, write factual, indisputable findings, and issue audit reports which stand 
by themselves, without need for oral expanatory statements by the auditors. Contrary 
to popular belief, auditors are not God - but, through the strength of knowledge and 
professional conduct, they can be a major influence on and benefit to the entire QA 
effort, management, the organization, and the end product. 

I submit that it should be every auditor's fervent goal to some day be able to 
perform a thorough audit without finding a single significant finding. Although it has 
yet to be my personal experience to achieve such a revelation during an audit, I have 
heard the occasional isolated sounds of an auditor who has entered such pearly gates. 
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The notion that an auditor must have findings to justify his job is unacceptable 
to me. Auditors found to exhibit this personal philosophy should be exorcised from the 
program, auditees who fear such an approach should be counselled and assured to the 
contrary; and, programs which endorse such a backward concept should be scrapped, 
together with its misguided management. 

PROGRAM'S BENEFITS 

There should be little debate on the regulatory necessity for independent review 
of quality affecting activities through audits. The need to continually assure the 
protection of the health and safety of the public is a responsibility which cannot be 
overemphasized. It is vital in maintaining nuclear power as a viable, safe form of 
energy. However, I believe there are additional motivators for enacting a thorough, 
well-managed, professional QA auditing program. 

One of these motivators is the realization that often the only direct interface a 
contractor or vendor has with the client, is during client's audits of their activi- 
ties. This could be true whether it is a reactor vendor auditing one of its suppliers, 
an architect-engineer auditing a vendor, a utility auditing the architect-engineer or 
the Nuclear Regulatory Commission inspecting the utility. The potential impact of 
this "flag waving" opportunity is significant, since the image of the auditing organi- 
zation could hinge almost entirely on the manner in which the audit team executes its 
responsibilities. The extent to which the auditors are professionals could affect the 
attitude of the audited company toward the project as a whole. Immeasurable harm or 
benefit can be derived from the experience - I have seen both. The positive reaction 
has never failed to impress and please me. The negative situations always make me 
want to shake the auditor by the shoulders to reemphasize the impact his actions have. 

An effective audit can cause the audited organization to change its attitude in 
addition to its behavior. This is most important, for behavior only reflects what we 
do - attitude reflects how we feel about what we do - one's commitment, if you will. 

An effective audit can also place emphasis on what is important, and thereby reduce 
the amount of effort or attention being placed on extraneous or insignificant activi- 
ties - activities which are not contributing toward the quality of the service or 
product. 

A very important benefit of a comprehensive, well -planned, fully executed and 
documented program is its capability to satisfy the third party auditor who is audit- 
ing, inspecting or surveying your company. 

If your own internal auditing efforts are weak or incomplete, expect the third 
party auditor to penetrate deeper into your company's activities. If your external 
audit efforts are inadequate, expect the third party auditor to consider accompanying 
you on an external audit, having you increase your audit frequency, and/or having 
other incremental impacts on your program. Conversely, a sound QA auditing effort 
will establish credibility with the third party auditors, resulting in more favorable 
reports on the company's management and activities; a very positive cause-and-effect 
relationship. 

Management’s satisfaction with a credible program may lead to the auditor's in- 
volvement in investigating "those sensitive" situations which occur from time to time 
in an organization. Whether this involvement takes the form of specific participation 
by the QA auditors or simply using their techniques, it further demonstrates that man- 
agement perceives the QA audit process as an effective management tool which can be 
tailored to investigations, whether they be QA-oriented or not. The use of auditing 
techniques in typically non-QA areas is a vital theme. A proven auditing system can 
result in other organizational components emulating key features of the program, such 
as planning techniques, reporting formats, scheduling methods, briefing session 
philosophy, status and commitment tracking, etc. This form of flattery will increase 
the level of understanding and acceptance of the overall QA program, thereby strength- 
ening the QA effort. 


FLEXIBLE SYSTEM 

Auditing is a flexible technique and as such can be easily applied to a variety of 
areas. Examples of this application are financial auditing, operational or management 
audits, administrative reviews to determine the facts surrounding particular situations, 



whether they be EEO discrimination claims, potential personnel disciplinary cases, 
environmental infraction allegations, safety investigations or other fact finding mis- 
sions. 


The Comptroller General of the United States notes in the forward to a pamphlet 
entitled "Auditors Agents for Good Government," that "Auditing is often considered... 
to be primarily concerned with the proper safeguarding of funds and property. Actually, 
this is only part of the concern of auditing. The Comptroller General goes on to 
state that. .. "Auditing can be very useful to those... who recognize its potentials and 
use its reports. It can, for instance, alert them to potential problems so that they 
can make programs work effectively and correct inefficiencies and uneconomical prac- 
tices before serious or even irreparable harm is done."^ 

During my association with the nuclear industry, I have seen audit concepts suc- 
cessfully function in such non-QA related situations as a review of conflict of inter- 
est charges, fraud allegations, assessment of cause and responsibility for a fish kill, 
compilation of facts regarding contract disputes, determination of need for specific 
management systems refinement, and so on. 

Additionally, I have seen audits effectively performed on such non-routine QA sub- 
jects as a review of disagreements between surveillance agency personnel and the per- 
sonnel of the manufacturing company being surveyed, a review of statements quoted in 
newspaper articles by contractor's personnel regarding QA, reviews of questionable 
material suitability charges, and others. 

These experiences clearly indicate that audits are flexible, viable options which 
management can, should and frequently does rely on to determine the effectiveness of 
its operations. 


SPECIFIC EXAMPLE 

To further emphasize the usefulness of this management investigative tool, I 
would like to cite one example in particular. The case in point is the North Carolina 
Utilities Commission ordered audit of Carolina Power & Light Company (CP&L). This 
audit, or study, was an eight-month evaluation of CP&L's management systems, procedures, 
and performance. CP&L, a company whose hallmark is a progressive style, open communi- 
cations and responsive management, issued a pamphlet entitled "What Kind of Job is 
CP&L Doing?" This document highlights the management performance study which they had 
undergone by an independent, outside consulting firm and provided its' customers, 
shareholders and employees with a concise, accurate summation of a 700 page report. 

In the pamphlet, the purpose of the audit is stated as follows: 

"The objectives of the study, as stated by the auditors, were to provide an im- 
partial, professional assessment of CP&L's management and operating efficiency, 
to identify opportunities for improvement, and to provide recommendations for 
achieving improved, more cost-effective operations." 3 

Commenting on the report, CP&L's chairman notes in the preface to the pamphlet: 

"The results of the audit show that CP&L is operating efficiently and that 
we are doing a good job for our customers. As in any large organization, 

CP&L has areas for improvement. Where the potential for improvement has 
been identified, we have initiated appropriate action."^ 

The pamphlet concludes by noting: 

"The report pointed out that CP&L has continued to be responsive to major 
changes - in economic conditions, in regulatory climate, in technology - 

^ Audi tors Agents For Good Government , Audit Standards Series Number 2, United States 
General Accounting Office, United States Government Printing Office, Washington, D.C., 
1973, p. i. 

2 

Loc. cit. 

3 

What Kind of Job Is CP&L Doing?, Carolina Power & Light Company, Raleigh, 1977, p. 2. 

^ Ibid . , p. 1 . 


and has been able to modify its organizational structure and operating functions, 
sometimes quite drastically, to adapt to these new conditions." 5 

In summarizing the findings, the outside auditors observed: 

"The interest in and resolve of the top management during the course of the 
study in moving directly and promptly on opportunities for improvement pro- 
vides assurance to the consul tants fi and the commission of CP&L's continued 
resolve to carry out these plans." 

The example of CP&L underscores the concept that audit programs can provide man- 
agement with the opportunity to assure that vital activities are being performed sat- 
isfactorily, efficiently and in accordance with commitments, thereby maintaining the 
company's internal and external credibility. 

MANAGEMENT PRINCIPLES 

Another way of realizing the value of the audit function to goal -oriented manage- 
ment, is to select a few key phrases from criterion 18 of 10CFR50, Appendix B entitled 

"Audits". This approach results in phrases like "A comprehensive system to verify 

compl iance. . . .and to determine effectiveness Audit results shall be documented 

and reviewed by management " Aren't these the basic managerial principles of 

monitoring against goals, feedback, follow-up and management action? 

In essence a QA program is a comprehensive management system, with audits repre- 
senting certain basic elements of the overall management process. The concepts of 
planning, defining authority and responsibility, setting goals, integrating implemen- 
tation, measurement, feedback and corrective action are all represented in an effec- 
tive QA program; and can be reviewed through an auditing function. 

Certainly audits are time consuming. But as the venerable Dr. J. M. Juran notes 
regarding audits in his book entitled "Quality Planning And Analysis," "All this takes 
time. This is the price one pays for the realism of facts over the drama of opin- 
ions . "8 


SUMMARY HIGHLIGHTS 

In summary, to develop a useful, comprehensive QA Auditing Program, three funda- 
mental ingredients are required: 

• Credible QA personnel to advance program. 

• Top management support. 

• Selective hiring of QA audit staff. 

Given the establishment of a well planned, professionally staffed QA Audit effort, 
program benefits can be realized in several areas such as: 

• Owner's successful representation in outside firms. 

§ Improved attitude and behavior by others toward QA. 

• Emphasis focused on salient points. 

t Increased credibility with third party. 

• Audit techniques used in non-QA areas. 

In conclusion, I content that the limits to which QA auditing as a system are of 
benefit to management is quite simply a function of the credibility and resourceful- 
ness of the QA people, and the commitment and perception of their management. 

5 1 b i d . , p. 9. 

6 Loc. cit. 

7 Qua1ity Assurance Criteria For Nuclear Power Plants and Fuel Reprocessing Plants , 
Title 10, Code of Federal Regulations, Part 50, Appendix B, General Services Adminis- 
tration, Washington, D. C., 1977, p. 301. 

8 J. M. Juran and Frank M. Gryna, Jr., Qua lity Plann i ng and Analysis , McGraw-Hill Book 
Company, New York, 1970, p. 563. 
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GENERAL PURPOSE A.T.E.’s ARE NOT FOR EVERYONE 


J. Richard Lucas 
Instrumentation Manager 
Honeywell, Inc. 

Process Control Div. 

Ft. Washington, Pa. 19034 


INTRODUCTION 

The advent of computers and microprocessors have brought a tremendous expansion 
in the types and capabilities of automatic test equipment (A.T.E.). A.T.E.’s are 
sought after because they provide cost savings: 

« Repeatable Testing 
« Minimum Operator Training 
» Faster Testing 
« Faster Diagnostics 
f More Complete Testing 

The computer directed general purpose A.T.E. provides printed circuit board 
(P.C.B.) manufactures with flexable and repeatable testing means; but is it what 
he needs? The following discussion deals with this by describing the basic types 
of A.T.E.’s, a review of potential pitfalls, and outlining a method for equipment 
selection. 


TYPES OF PCB A.T.E.'s 

Before an ATE is selected, a review of the basic types of ATE’s will provide 
a prospective on the options available. 

1. General Purpose A.T.E. - Tests digital, analog and hybrid assemblies. 

Digital diagnostics is directed to a node or a 
component . 


Analog diagnostics usually to a functional area. 

Programming usually through the aid of circuit 
simulation for digital and manual for analog. 


Investment : 


Advantages : 


Disadvantages : 


Machine $200K to $300K 

Fixtures $1.K to $5K 

Programming 80 to 160 hours 

1. Capability of testing a wide 
varity of PCB’s (Digital, 
Analog, Hybrid) 

2. Fully directed testing and 
diagnostics which results in 
minimum operator training and 
skill. 

3. Measurable test completeness or 
comprehension. 

1. High investment in hardware and 
software. 


2. Due to machine flexibility, 

extra caution must be exercised 
to prevent production bottlenecks 
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Sources : (1) 


1. Computer Automation, Gen Rad, 
Hewlett-Packard . 

2. Incircuit Tester Tests for shorts, opens and component function on the 

PCB through a "bed-of-nails” fixture which accesses 
many or all of the interconnection points. 

Diagnostics to the component. 

Programming is done with automatic program generation 
with manual timing. 

Investment: Machine $60K to 90K 

Fixtures $ 7K to 1.5K 
Programming 20 to 60 hours 

Advantages: 1. Low fault isolation cost 

2. Low programming cost 

3. Excellent for bring up yield on 
a more expensive functional 
tester. 

Disadvantages: 1. Limited functional testing 

2. Complex fixture generally re- 
quired for each PCB. 

Sources: (1) Faultfinders, Testline, Zehntel 

3. Digital Comparison Tests a digital PCB by comparing the boards reaction 

Tester to stimuli with that of a known good board. 

Digital diagnostics to a component or node. Requires 
an operator to use a logic diagram. 

Programming is done by manually selecting pulse 
sequences on a known good board. 

Investment: Machine $15K to $70K 

Fixture $1K to $ 3K 

Programming 40 to 80 hours 

Advantages: 1. Low initial investment 

2. Fast PCB screening capability 

3. Loxtf cost programming 

4. Tests can be performed at speed 

Disadvantages: 1. Ho analog 

2. Technically competent operator 

to isolate faults. 

Source: (1) Fluke 

Custom built equipment using a microprocessor to test 
specific board types. Typically developed by the 
manufacturing house to fill a specific volume of test 
requirements. 

Programming is manual 


4. Dedicated Automatic 
Tester 


Investment : 


Machine 

Fixtures 

Programming 


$15K to $30K 
Included in above 
20 to 120 hours 



Advantage : 


Disadvantage : 


1. Custom built to provide 
accuracies that are not available 
in commercial equipment. 

2. Repeatable test for complex PCB 
(particular importance in a long 
step by step test sequence in 
which the operator could omit 

a step) . 

3. Low cost if it potentially saves 
the investment of another 
General Purpose A.T.E. 

1. Equipment life generally tied to 
product life. 


The purpose of the above listing was to show the wide variations of machines that 
are available for consideration. One, some or all of these might have a place in 
your manufacturing line. 


(1) Sources are not meant to be all inclusive but suggest typical manufacturers. 

A.T.E. PITFALLS 


There are an infinite number of problems that one can get into when using ATE 
Technology. However, for your consideration, a brief listing has been prepared of 
common problems which if planned for can be controlled. Without planning, the re- 
sults can be painful to the unsuspecting user. 

1. The first and one of the most significant problems can be best described as "After 
spending $XXX thousand, it won’t test more than three card types in the first six 
months"? It must be remembered that a computer directed tester is only as smart 
as the program. This leads you to the Instrumentation Programming Engineer who 
develops the software. This manpovjer must be provided to get the programming 
results in the time expected. Using an outside source, such as many of the ATE 
producers provide, does not fully solve the software problem, but it is excellent 
for helping with peak loads. Considerable coordination on what is wanted in the 
program is required between the outside source and your in-house programming 
engineer. Thus, machine utilization may be low unless adequate software plans 

are made at the time of equipment purchase. 

2. The next problem is called the "bottleneck". This is the description of what 
happens when your production flows through the ATE and due to machine downtime, 
testing volume, or programming debugging load, there are not enough hours avail- 
able to accomplish that which is required. There are two major contributors to 
this pitfall: 

a) Being so excited with the capability of the new machine, one starts putting 
everything possible on it. The result is as the volume picks up, the machine 
becomes loaded with work that in many cases should be done by lower cost 
dedicated testers (manual or automatic). 

b) Don’t place on a general purpose ATE one or two PCB’s that consume most of the 
machine time (due to volume and test time). This wastes the money spent to 
get versatility. Consider a dedicated ATE for cases of this nature. 

3. This one is affectionately known as "make it work but don't touch it". The 
interpretation of this statement is one test program is not working but the 
engineer can not get on the machine because production is busy testing other 
needed units. This problem becomes more dangerous as more and more programs are 
operating on the ATE. It must be remembered that besides software development, 
machine time will be required for periodic production problems due to machine 
failure or components from a new vendor causing unexpected results during 
testing. The control here is to forecast workload and establish production 
testing on second and third shifts. Also, an off line software development 
machine (a standard machine less PCB drivers receivers or stimulas sources) can 
also help lighten the load on the production ATE. 
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4. The last of the Insidious problems is "now that the PCB is rejected, what do I 
do with it"? Sometimes the test engineer would like to tell production but the 
real solution is to provide manual diagnostic equipment. This is not always 
necessary but should be kept in mind for diagnostics that lead you to an area 
rather than a specific problem. 

A.T.E. SELECTION 

When determining what type of ATE is needed, a critical review of what are the 
test requirements must be done. The following list should be included in any 
effective review plan; 

1. Estimated Product Life - Short product life will mean an ATE investment is not 
practical. Where long life is expected, the product testing and diagnostics 
investment can be high and still return excellent savings. 

2. P.C. Board Type - The types (Analog, Digital, Hybrid) of PCB to be tested must 
be known so that machine capability can be determined. 

3. P.C. Board Volume - This permits establishing priority of which gets programmed 
first because of payback. Also, where the PCB gets tested. Remember the pitfall 
of one or two cards tying up a general purpose ATE. 

4. Product Outgoing Quality - This is the consideration of the consequences of a bad 
PCB going undedicated to the next higher level of assembly. In some cases , a 
very complete second level test is performed and if faults get through, they can 
be detected. The feedback may be used to tune the PCB test program to improve 
the test comprehension. However, there are cases where ATE test is the final 
verification and direct shipment to the customer will occur. In these cases, 
careful consideration to the test comprehension must be done from the start. 

5. PCB Functional Requirements - The functional requirements of the PCB such as 
accuracy, number of tests to be performed, types of signals, etc. are needed 
for selecting the make-up of the A.T.E. 

6. Operator Interaction - Pot adjustments and switching are common operator inter- 
actions. These can waste ATE time and in these cases, it may be better to use 

a manual tester. A dedicated ATE could also be considered if the volume and 
test difficulty justify it. 

7. Test and Diagnostic Time - Both of these must be considered along with the 
expected yield so that an accurate estimate of machine loading can be established. 

Now that the test requirements have been gathered, the criteria for the test must 
be placed into "must" and "want" categories. The definition on the "must" should be 
as precise as possible because this decision says that when reviewing a piece of 
equipment, if it does not comply, it’s out. The "wants" should be weighted as to 
importance to your application. 

Attachment I shows an analysis used for the selection of a general purpose ATE 
for Honeywell, Process Control Division. This project started with a list of twenty- 
two equipment suppliers. These were quickly reduced to four vendors because the 
others did not meet all of the "Musts". 

Once the scope of the investigation has been brought down to a selected few, it 
is recommended that a representative PCB be given to the top candidates so they can 
demonstrate their capability. This approach provides a good prospective on software 
and hardware performance; how difficult is the machine to program and how does it: 
respond to the program. Care must be exercised in analyzing the software effort 
because next to the machine investment, the programming development is the most 
significant cost encountered. 

After witnessing each vendors performance, a KT(2) chart should be prepared 
(Attachment II). Each attribute being considered should be weighted based on its 
importance to your specific needs. In this example the weight ranged from 2 to 10, 
where 10 was the most and 2 was least important. Then each vendor should be rated 
on how well his machine fulfils each attribute. A perfect rating was 10 and poorest 
rating was 2. 

(2) The Rational Manager by Charles Kepner and Benjamin Tregoe 



Thus the weighted performance for each machine is the product of the attribute 
weight times the machines weighted compliance with that attribute. 

Example Calculation for Cost Attribute: 

Weight - 6 (Selected as medium importance) 


Vendor 

Cost 

Best Price Performance 

A 

$236. 4K 

7 

B 

$310. 4K 

4 

C 

$292. 2K 

5 

D 

$312. 9K 

2 


Weighted Performance for Cost is: 

Vendor Weight 

Performance 


Weighted Performance 

A 

6 

X 

7 

= 

42 

B 

6 

X 

4 

= 

24 

C 

6 

X 

5 

= 

30 

D 

6 

X 

2 

= 

12 


This analysis is continued for all attributes considered. Then each machine 
vendors volume is totaled. In the example in Attachment IT, vendor C with 495 points 
did the best job of meeting all the attributes. 

The attached flow chart (Attachment III) shows what Honeywell PCD does in their 
testing of the newly released Total Distributed Control (TDC-2000) product line. 
P.C.B.’s are assembled, wave soldered and then pre-screened on an incircuit tester 
for shorts, opens, and component problems. The cards then move to one of five 
different testers depending on the testing requirements for the board. There is also 
a system test which provides feedback on how effective the PCB testing has been. 

CONCLUSION 

This review showed the various testers available and depending on the specific 
manufacturing needs, the general purpose ATE is not for everyone. The decision 
of how to test is primarily one of economics - cost of equipment, cost of fixtures, 
cost of programs, cost of finding a failure at a later level of testing. There is 
no one correct answer. The best solution is that which fits the manufacturing needs 
of your operation. 
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ATTACHMENT I 

GENERAL PURPOSE A.T.E, ANALYSIS PQR. HONEYWELL , PCD, FT, WASHINGTON, PA. 
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ATTACHMENT II 


K-T ANALYSIS OF A.T.E. 

FOR 

HONEYWELL FT. WASHINGTON, PA. 
VENDORS 


A B C , D 



WEIGHT 

REL 

EXT 

REL 

EXT 

REL 

EXT 

: REL 

EXT 

COST 

6 

7 

42 

4 

24 

5 

30 

2 

12 

ANALOG DC MEAS . 

8 

6 

48 

7 

56 

4 

32 

4 

32 

ANALOG AC MEAS. 

4 

8 

32 

4 

16 

6 

24 

5 

20 

ANALOG STIMULUS 

7 

6 

42 

6 

42 

3 

21 

5 

35 

NODAL ACTIVITY 

7 

9 

63 

4 

28 

6 

42 

4 

28 

MODELING & SIMULATION 

9 

3 

27 

8 

72 

5 

45 

9 

81 

EDIT CAPABILITY 

5 

4 

20 

4 

20 

8 

40 

9 

45 

FAULT VERIFICATION 

6 

2 

12 

3 

18 

8 

48 

7 

42 

EASE OF MAINTENANCE 

8 

5 

40 

2 

16 

4 

32 

6 

43 

8 

WARRANTY 

2 

2 

4 

3 

6 

5 

10 

4 

UUT INTERFACE 

4 

4 

16 

5 

20 

6 

24 

4 

16 

COMPUTER & MEMORY 

3 

5 

15 

6 

18 

3 

9 

8 

24 

DIAGNOSTICS 

8 

5 

40 

8 

64 

6 

48 

7 

56 

PROVEN CAPABILITY 

10 

3 

30 

7 

70 

9 

90 

' 2 

20 

TOTALS 



431 


470 


495 


467 
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CALIBRATION SUPPORT FOR NUCLEAR POWER STATIONS 


David R. O'Brien 

Standards and Testing Facility Administrator 
Steam Production Department 
Duke Power Company 
Charlotte, North Carolina 28242 


INTRODUCTION 

Measurement support for Nuclear Power Stations is an economic and engineering 
necessity. Adequate calibration control assures that measurement of plant processes 
jare valid, accurate and within acceptable limits. To fulfill this need, Duke Power 
Company maintains a centralized standards laboratory which assures that company 
measuring and test equipment is accurate and based on either lawfully established 
units of measurement or physical constants of nature. Four nuclear stations are 
currently serviced by the laboratory: Oconee Nuclear Station (which is in operation), 
McGuire Nuclear Station, Catawba Nuclear Station, and Cherokee Nuclear Station con- 
sisting of 10 nuclear reactor units all located in North and South Carolina. 



FIGURE 1 Oconee Nuclear Station 


CALIBRATION ECONOMICS 

Economic operation of power generating units is a necessity to meet company ob- 
jectives of providing reliable power at the lowest practical cost to the consumer. 

Why, then, are calibrations and standards considered such an important area? Similarly 
stated, calibration errors can cost money. 

Duke Power Company now has three pressurized water reactors in operation at 
Oconee Nuclear Station in South Carolina. Each unit is capable of supplying 886 Mw 
of electrical energy. A primary consideration for efficient operation is the control 
of steam pressure and temperature at the turbine throttle. Typical increased oper- 
ating cost for Oconee Nuclear Station, based on a 1% loss in throttle steam tempera- 
ture and pressure, is illustrated as follows: 

(a) 1°F loss in throttle steam temperature from optimum results in $48, 000/year 

higher generating cost (3 units) 

© 1978 American Society for Quality Control 



(b) 9 PSIA loss in throttle steam pressure from optimum results in $115 , 200 /year 

higher generating cost (3 units) 

Type "J u (iron constantan) thermocouples are utilized to monitor temperature 
of throttle steam. Normally, such thermocouples have calibration uncertainties of 
2°F. Accuracy of the thermocouple, discounting calibration uncertainty, can approach 
5°F. The addition of reference junctions, thermocouple extension wires and selector 
switches can introduce another ±6.5°F of error. These figures result in a worst 
case condition of ±13.5°F uncertainty when utilizing normal state of the art measure- 
ing equipment. Steps can and are taken to minimize this figure. However, thermo- 
couple output does change with time. Often uncalibrated thermocouples can have errors 
considerably larger than those noted. Therefore, recalibration and comparison to 
known temperatures can and must be accomplished at regular and periodic intervals in 
order to reduce the measurement uncertainties and subsequent higher generating cost. 

Pressure transmitters are utilized to monitor throttle steam pressure and are 
usually accurate to ±1/2%. On recalibration, some transmitters have been known to 
have errors in excess of 1% of full span or range (120 PSIA). These figures are con- 
siderably larger than the 9 PSIA figure previously mentioned. Errors of this magni- 
tude must be reduced to acceptable limits. Again periodic recalibration is a must; 
however, it should be realized that no measurement is perfect. State of the art 
limitations in science and technology limit the accuracy of many measurements and 
control processes. 

Bearing failure can cost considerably more than replacement of bearings before 
failure. Normal maintenance before failure might require one or two days while a 
failure repair on the same equipment might keep a unit out of service for several 
weeks. Therefore, valid engineering decisions must be made based on accurate measure- 
ments. These decisions are often based solely on instrument measurements such as 
bearing temperature. If a thermocouple errorneously indicates excessive bearing tem- 
perature, a unit may be shut down prematurely with the subsequent reduction in system 
capability, unit availability, and increased operating cost. Conversely, a thermo- 
couple or resistance thermometer which indicates a satisfactory temperature, when in 
fact the temperature is excessive, might lead to major damage to equipment and in- 
ability to meet the consumers demand for electricity. Calibration gives the accuracy 
and assurance necessary for valid engineering judgement in these circumstances. 



FIGURE 2 Standards Laboratory Electrical Calibration Area 


REGULATION 

Nuclear Power Plants, of course, operate within the framework of many regulations 
and under the inspection of regulatory authorities. Title 10 Code of Federal Regula- 
tions, 10CFR50 Appendix B, establishes the Quality Assurance criteria for Nuclear 



Measures shall be established to assure that tools, gages, 
instruments, and other measuring and testing devices used in 
activities affecting quality are properly controlled, cali- 
brated, and adjusted at specified periods to maintain accuracy 
within necessary limits. 

Duke Power has developed a centralized standards laboratory which fully imple- 
ments this regulation and provides the necessary support to nuclear stations in 
the following measurement areas: 

- length 

- flow 

- temperature 

- pressure 

- electrical measurements (resistance, voltage, current...) 

- acceleration 

- mass 

- rotational 

- humidity 

Several thousand instruments, gages, and test equipment can be found at each 
nuclear reactor unit. Figure 3 illustrates the massive number of instruments found 
solely in the control room. Each instrument/display is fed by one or more sensors, 
all needing calibration. 

Other criteria that may be imposed in addition to the above, depending on the 
type of reactor include the American Society of Mechanical Engineers (ASME) Boiler 
and Pressure Vessel Code Section III, ANSI N45. 2-1971 or RDT-F-3-2. 



FIGURE 3 Nuclear Reactor Control Room 


CALIBRATION CONTROL 

A calibration system specification details in writing what measures have been 
implemented to control the calibration process. In addition to a system description, 
written calibration procedures are a necessity. The calibration specification util- 
ized by Duke Power Company encompasses each of the following areas: 

- Calibration Procedures 

- Calibration Records 

- Recall System 

- Calibration Intervals 

- Instrument Identification 


- Status Labeling 

- Environmental Controls 

- Calibration Service Procurement 

- Traceability 

- Nonconforming Instrument Control 

- Audits 

- Personnel Training 

- Accuracy Ratio 

- Standard to Test Instrument 


Procedures 


Written procedures are required for all activities affecting quality and are 
developed prior to instrument calibration. Procedures are either developed internally 
or when possibly extracted from the government industry data exchange microfilm data 
bank. After review and approval, these procedures are issued as controlled documents. 
Procedures that are developed are written to conform with the National Conference of 
Standards Laboratories recommended guide. Quantitative and qualitative acceptance 
criteria is included in the text of the procedure for each instrument. 

Records 


A master history file for each item of measuring and test equipment is generated 
and stored for as long as the station operates or the life of the instrument whichever 
is greater. The equipment history contains: 

- instrument identification number 

- name of instrument manufacturer 

- model number 

- manufacturer serial number 

- equipment description 

- calibration interval 

- purchase date 

- calibration history 

- specialized remarks 

- calibration data (as found and as left) 

- out of tolerance notices 

- calibration status 

Each instrument is assigned a unique identification number consisting of ten 
alphanumeric characters. The first five characters identify the nuclear station and 
group responsible for the instrument. This is the instrument master number and all 
records are keyed to this identifier. 

Traceabilit y 


Each instrument calibrated must have its calibration genealogy traceable through 
serialization of the instrument and documentation of this serial number on the cali- 
bration report, to a nationally recognized standard. This can be accomplished by 
directly returning certain standards to the National Bureau of Standards (NBS) or by 
having suppliers calibrate the standards with instruments calibrated by or traceable 
to NBS. Often return of the instrument to an outside agency takes a considerable 
amount of time. This demands increases in the inventory of measuring anil test equip- 
ment to assure continued operations. 

If possible, one way of minimizing this extra inventory and reducing cost is to 
provide calibrations based on natural physical constants. This calibration method 
is accomplished by utilizing light waves as a standard of length and melting points 
of selected metals and the triple point of water for temperature. A great deal of 
the measurement uncertainty associated with transfer measurements of the comparison 
type can be eliminated and more accurate measurements realized when using natural 
physical constants. However, at some time during the calibration cycle, comparative 
transfers must be accomplished. The standard is maintained at four times the accur- 
acy requirement of the measuring and test equipment calibrated. This results in 
the so-called "4 to 1" rule which reduces the uncertainty associated with the cali- 
bration process. State of the art technology often precludes the maintenance of 
this "4 to 1" rule with a resulting decrease in accuracy. 



Mathematical and statistical methods are employed which result in a scientific 
analysis of error uncertainty associated with the calibration process when this 
"4 to 1" rule is not achievable or infeasible. Carefully planned experimental proce- 
dures and repeated measurements result in adequate control when limitations on accur- 
acy are encountered. 


COMPUTER DATA BASE 

A computer data base is maintained which stores a current history of the instru- 
ment via a Cathode Ray Tube (CRT) terminal which is linked to an IBM 370-165 computer 
system. This terminal is used to access and update this computer data base both at 
the laboratory and at the generating stations. While the program is an invaluable 
aid and provides for efficient management of several thousand instruments, it is not 
a quality assurance requirement. 

The equipment history data base contains data such as: 

- identification number 

- model number 

- manufacturer 

- equipment name 

- calibration interval 

- serial numbers of instruments used to calibrate the instrument 

- calibration date 

- calibration status 

- calibration due date 

- calibration schedule date 

Traceability is assured by the entry of the identification number of the test 
standard into the data bank. The computer checks to assure that the standard used for 
the calibration has a current calibration status traceable to the National Bureau of 
Standards by scanning the data bank. If the standard used does not have a valid 
status, the computer changes the instrument entered to a "REJECT” status. Standards 
discovered to be in error can cause a bookkeeping problem since the effect of out-of- 
tolerance conditions on prior calibrations must be evaluated. A cross reference pro- 
gram can be initiated that will list all instruments calibrated by the non-conforming 
standard since its last valid calibration. 

Recall of instruments occurs automatically as the computer generates a daily 
list of all instruments that have exceeded their specified calibration interval. The 
computer changes the status to "out-of-calibration" in the data base. Additionally, 
an Instrument Management Program is generated weekly listing a summary of each instru- 
ment's status as illustrated in Figure 4. 

SUMMARY 

Efficient and reliable operation of nuclear power stations is a must. Calibra- 
tion provides one mechanism to support this effort. Accurate measurements lead to 
valid decisions. Calibration error may be causing increased generating cost and 
inefficiency. Industry standards and regulatory reqquirements demand calibration to 
assure the efficient and reliable operation of power reactors. Measurement control 
systems, as described, allow for accurate engineering assessment of test and oper- 
ating data to assure efficient and reliable operation of nuclear power plants to 
supply our customers demand for energy. 
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SYSTEMATIC MEASUREMENT ERRORS 


Rolf B. F. Schumacher, Metrology Laboratory 
Rockwell International, Anaheim, California 


I. INTRODUCTION 

Statistical analysis of measurement errors has become an additional tool metrologists are 
using in an increasing number of applications and at an increasing rate in order to determine 
numerically the accuracy of measurements and the confidence attached to quoted accuracies. 
Methods of evaluating random errors are well known and in widespread use. This paper exam- 
ines the main aspects of systematic errors and methods to evaluate them; it discusses the nature 
of die fluctuating boundaries between systematic and random errors in measurements and 
calibrations. 

The systematic error of a measurement or calibration may be defined as the largest possi- 
ble estimated difference between the true value of a measured quantity and the mean value 
towards which the measurement or calibration process tended as a limit at the time of the meas- 
urement and which difference could not be eliminated for technical or economic reasons; it is an 
estimate of the maximum limits of the effects of all error sources known or suspected to exist 
which tended to offset uniformly all results of repeated applications of the same measuring or 
calibration process at the time of the measurement. Thus, the "systematic error" is not an 
error in the accepted sense of the word "error", but rather an uncertainty; however, the term 
"error" will be used here for brevity and because it has been firmly established by custom. 

II. TYPES OF MEASUREMENT ERRORS 


A. General 

The error or uncertainty of a measurement can be divided into a fixed part and a variable 
part, where it should be understood that the fixed part remains approximately fixed only for the 
duration of the measurement process. For shorter durations, the variable part is likely to be 
smaller, and over longer time intervals, the phenomena causing the "fixed" error can be 
expected to vary also, thus making the variable part of the error larger at the expense of the 
"fixed" part. As the time interval under consideration becomes very large, any initially fixed 
error will probably vanish and all errors are likely to become variable. 

The errors or uncertainties of a measurement can also be categorized by a random part 
and a systematic part. The random error is part of, but not necessarily identical with, the var- 
iable error; and the systematic error includes the fixed error, but may also include a portion of 
the variable part of the error. 

All determinable errors which occur during one measurement process and which do not 
appear* to be part of the random error must be accounted for, otherwise the process may not be 
under statistical control. But "... until a measurement operation . . . has attained a state of 
statistical control it cannot be regarded in any logical sense as measuring anything at all. "( 4 ) 
Capability of control means that either the measurements are the product of an identifiable sta- 
tistical universe or an orderly array of such universes, or if not, the physical causes prevent- 
ing such identification may themselves be identified and, if desired, isolated and suppressed.’^ 1 ) 

B. Systematic Errors 

A systematic error is that value which, when added to, or subtracted from, the limiting 
mean value of a measurement process of a quantity, produces a range in which the "true value" 
of that quantity is believed to lie. We hope that the systematic error is always equal to or 
larger than the difference between the "true value" of the quantity and that value toward which 
the measuring process of the quantity tends, this tendency not being caused by chance fluctua- 
tions of any part of the measurement system. 

The systematic error is a bias, but generally unknown in magnitude and direction, because 
the "true value" is an abstract concept which cannot be physically realized. More pragmatically 


*Tl.a word "appear" is used, ". . . because, as is always the case in trying to find a law con- 
trolling a phenomenon, we can never be sure that we have discovered the law. Obviously such 
appearance is not sufficient in the logical sense although it must be in the practical sense. "O- 1 ) 
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". . . the systematic error ... of a measuring process refers to its tendency to measure 
something other than what was intended. . . On first thought, the 'true value' of the magnitude 
of a particular quantity appears to be a simple straightforward concept. On careful analysis, 
however, it becomes evident that the 'true value' of the magnitude of a quantity is intimately 
linked to the purposes for which knowledge of the magnitude of this quantity is needed, and can- 
not, in the final analysis, be meaningfully and usefully defined in isolation from these needs, "v*) 

A systematic error is always an estimate of a range, and estimating it requires a profound 
understanding of the measurement process if a realistic figure is to be arrived at. A quoted 
figure for the estimate of a systematic error means that the metrologist believes that he would 
have detected - and, therefore, been in a position to reduce - any error larger than that quoted 
as the systematic error. It implies that the actual, unknown error could be anywhere within that 
range. And ''. . . the wiser and more careful the experimenter’s search for systematic errors, 
and the more completely he has eliminated them, the less likely is it to lie near the limits of the 
range. "( 3 ) 

Dorsey^ describes the concept of the systematic error as being ". . . used to cover all 
those errors which cannot be regarded as fortuitous, as partaking of the nature of chance. They 
are characteristics of the system involved in the work; they may arise from errors in theory or 
in standards, from imperfections in the apparatus or in the observer, from false assumptions, 
etc. . . . They are frequently called 'constant errors,’ and very often they are constant 
throughout a given set of determinations, but such constancy need not obtain . . . 

"In searching for systematic errors, the logical procedure is to make a series of meas- 
urements, then to change something and to make another series, and to compare the means of 
the two groups. This will be repeated as often as may seem necessary. 

"In the absence of such a search, the worker can do no more than hope that all is going 
well. The fact that he sees no reason for suspecting the presence of an unknown systematic 
error is of no importance at all, no matter who the observer is. The really troublesome errors 
are exactly those that are not suspected. The expected ones can usually be to some extent 
eliminated. "( 3 ) 

Sometimes economics dictate the use of a coarse measurement system whose major com- 
ponent of systematic error can be determined by comparison with a more accurate measure- 
ment system. This component can then sometimes be accounted for by a correction. The 
remaining systematic error will thereby be reduced; it may even become negligibly small, but 
it will theoretically never become zero. However, the systematic error of the coarser system 
may vary, and it may not be practical to determine it each time a measurement is to be made. 

In this case, no correction may be applicable, and the comparison with a more accurate system 
may only serve to estimate the range of the systematic error of the coarser system more accu- 
rately than would be possible without recourse to a more accurate system. This is the case, for 
instance, with a working instrument with a rated accuracy or uncertainty of 1% routinely calibra- 
ted by working standards with 0. 1% uncertainty limits. The 1% uncertainty of the working 
instrument is its systematic error contribution to all measurements made with it, unless cor- 
rections are provided, in which case the systematic error may be less. The calibration against 
the standards then serves to assure that this systematic error does not exceed 1%. 

III. THE CHANGING NATURE OF ERRORS 

The boundaries between systematic and random errors are fluctuating. What appears as a 
systematic error under one set of circumstances may appear as a random error under another 
and vice versa, just as a sculpture changes its appearance to a viewer walking around it. The 
consequences of this fact, as they affect the evaluation of uncertainties of standards in a labora- 
tory, are illustrated by a few examples. 

A. A Random Error Becoming Systematic Error 

When a standards laboratory A, say the National Bureau of Standards, calibrates a refer- 
ence standard for us, laboratory A reports to us the value of the standard, as it has determined 
it by its measurements, and the limits of uncertainty of that measured value. Part of laboratory 
A's uncertainty originates from its random error, part from its systematic error. Random 
error and part of the systematic error can be expected to add a small variable component to 
laboratory A’s measured value in such a way that each time laboratory A measures the standard, 
the resulting measured values will differ slightly from the ones obtained earlier or later. How- 
ever, once a value and its limits of uncertainty are reported to us, they are fixed, no more 
variable, least of all "random variables", and the entire uncertainty, including random error, 
must be considered by us as a systematic error when we need to apply it. The necessity of the 
foregoing was discussed in detail by Youden in Reference 12. (See also Ref. 10.) 



In Part in.B. , under "varying Systematic Errors of Standards", we shall see how trend 
charts can help us "unfreeze" the random error and the variable part of the systematic error of 
laboratory A in the long run to reduce the total uncertainty accompanying the value of our stan- 
dard. But without such techniques, the total uncertainty of a higher echelon measurement 
becomes a systematic error in its lower echelon application. (See also Reference 15. ) 

B. Varying Systematic Errors of Standards 

The certification uncertainty of our standard is not the only uncertainty introduced by the 
standard. In general, another uncertainty term must be found which combines all those fluctua- 
tions in the value it represents as may be caused by changes in the standard itself, usually due 
to drift, instability, or use (wear), or by external influences which temporarily affect the value 
of the standard, such as temperature, barometric pressure, air ionization, solar activity, etc. , 
and which are not controlled - and may not even be known to have an effect — at the time the 
value of the standard is being determined. Some of these effects may at least partly be 
accounted for in the random error of the measurement process employed when determining the 
value of the standard. But, unlike that part of the random error which is entirely caused by the 
measuring process and the effect of which will remain constant once the measuring process is 
terminated and the value of the standard reported, the latter effect will continue to change the 
value of the standard, adding to the uncertainty. 

In general, we would not know whether differences in subsequent values of our standard as 
certified by Laboratory A are caused by the random error of Laboratory A only, by changes in 
Laboratory A's systematic error, by changes in the value of the standard itself, or by a combi- 
nation of these causes. To be able to assign to a standard an uncertainty which includes all 
experienced effects of variation from the unknown and unknowable true value of the standard, we 
must analyze the history of that standard. 

A valuable tool in analyzing and displaying the history of a standard is a Trend Chart (see 
Figures 1 and 2). The dots and x T s on the Trend Charts represent values reported by the 
National Bureau of Standards (NBS) for the standards in question at the indicated dates. Their 
dispersion pattern suggests a Trend Line which represents the best value known of each standard. 
The Trend Line value of the standard is most likely a more accurate value of the standard than 
the latest NBS reported value, and the Trend Chart has on occasion served to correct errors in 
the NBS reported values. The dispersion of the points shows graphically the effect of all the 
variables influencing the known value of the standard over the long run, such as possible long 
term variations in the systematic error of the standardizing Laboratory (NBS in this case), its 
random error, and actual changes of the standard. In fact, the Trend Charts of the gage block 
indicated a shrinkage of the block before the reasons for such shrinkage were known (wear 
effects could be excluded in the case of the block represented by Figure 2). 

Furthermore, the calculated envelope around the points at two or three sigma of the points' 
dispersion around the Trend Line represents the total uncertainty of the value of the standard, 
excluding the systematic error of the standardizing laboratory. The systematic error of the 
standardizing laboratory is possibly in part included in a Trend Chart spanning a long period, 
but we don't know how much of it is included, if anything. Thus, to estimate the total uncer- 
tainty of the standard's value at any time, we must add the reported systematic error of the 
standardizing laboratory to the width of the envelope on the Trend Chart. And this total uncer- 
tainty of the standard is part of our systematic error when we use tlie standard as a basis for 
measurements. Note that this systematic error includes random variables which have been 
frozen for the time being and must be considered as being part of the systematic error. 


For a detailed description on how to construct Trend Charts, see References 8* and 9. 
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Figure 1. Trend Chart of a Thomas Type Standard Resistor 


♦Reference 8 should only be used with an errata sheet available 
from the author since it contains numerous typographical errors. 


Figure 1 shows the 
Trend Chart of a standard 
resistor constructed in 
January 1965 (dashed 
lines). At that time, the 
best value for this resist 
tor was 1. 000 013 44 ohms 
(Trend Line) . In the 
absence of any informa- 
tion to the contrary, the 
Trend Line was believed 
to be horizontal, assum- 
ing no detectable change 
in the value of this resis- 
tor for the near future 
(two to three years). 


Because of the scarcity of the points 
available, 2-sigma control limits were 
drawn based on the scatter of the points 
around their average (the Trend line in 
this case); this gives a 95 percent confi- 
dence that the "true value" (i. e. the limit- 
ing mean value of all experienced vari- 
ables) lies between those control limits, 
disregarding for the moment any system- 
atic error of NBS. As more poinls 
become available 3-sigma control limits 
would reduce the risk of the limiting mean 
value being outside the control limits to 
an insignificant 0. 3 percent. 


As the Trend Line value of this 
standard is the best known value, this is 
the value assigned to the resistor at any 
time, regardless of the latest calibration 
value of the resistor reported to us by 
NBS. Note that the reported values have been jumping up or down by as much as one part per 
million from one calibration to the next. If we would always assign the latest calibration value 
to these standards, our entire measurement system would experience these ripples, ripples 
which in the end would cancel each other around the limiting mean value, the Trend Line value. 
Instead, we compute a new Trend Line value, a new mean value in this case, every time a new 
calibration value is obtained. Thus, every new calibration value will change the new mean only 
insignificantly, if at all. Hence the Trend Line value assigned to the standard will remain stable 
over the years. 

The control limits represent the limits of uncertainty of the Trend Line value and hence, 
represent the limits of uncertainty due to variable error components of the value which we assign 
to the standard. They encompass many more variables and sources of uncertainty than those 
experienced by NBS in the calibration of the standards. The limits of uncertainty about the 
Trend Line of the first five values of Resistor A, Figure 1, are at 0. 65 microohms from the 
Trend Line; adding to that the estimated limits of the systematic error by NBS of about 0. 5 
microohms, the total uncertainty of the value of Resistor A (1. 000 013 44 ohms) is 1. 15 micro- 
ohms. This is the systematic error introduced by the standard to any measurement made with 
it. Note that the 0. 65 microohm control limits represent a 95 percent confidence that this range 
brackets the "true value" of the standard. If we want to increase the confidence limits to 99. 73 
percent (3-sigma equivalent), the control limits would have to be widened to 1.53 microohms 
about the Trend Line value for a total uncertainty of 1. 53 + 0. 5 = 2. 03 microohms. In order to 
narrow these limits of uncertainty, we need a longer history with more values. 

Many a laboratory is using the latest calibration value of their standard as the value 
assigned to it until the next calibration of that standard and use the uncertainty reported by the 
calibrating laboratory (NBS in our case) as the total uncertainty of the standard's value. The 
fallacy of that practice becomes evident from die above discussion. These laboratories in fact 
experience a far higher uncertainty than they are aware of. 

Figure 2 shows a Trend Chart of a special gage block to illustrate the fact that some stan- 
dards undergo gradual changes in time. In such cases, a Least Squares Trend Line can be 
constructed, depicting with sufficient accuracy the behavior of the standard and permitting a 
forecast of its most likely value for the near future. Since the slope of such a Least Squares 
line is also burdened with some uncertainty, the band of uncertainty around the forecast future 
values of such standards continues to widen as time progresses as shown in Figure 2, express- 
ing numerically our uncertainty about the exact value of the slope. This widening bandwidth of 
uncertainty also tells us when we need to have the standard recalibrated, lest its uncertainty 
grows intolerably large for the requirements of our products. 

The dots represent the values known at the time at which the Trend Charts were construc- 
ted and on which the Trend Charts are based. The x represents a value obtained later, and is 
shown to illustrate the predictive value of the Trend Chart. The Trend Charts are, of course, 
updated each time a new value is obtained, as exemplified by the solid lines in Figure 1. 

C. Varying Systematic Errors of Measurements 

The magnitude of a systematic error of a measurement process is established by the 
capability and willingness of the metrologist to measure those phenomena which cause system- 
atic errors, to find their quantitative effects (i. e. what quantity of the phenomenon is required 
to cause a measurement error of a given magnitude), and to account for them or eliminate them. 




If it is known how much of a phenomenon causes a given amount of a measurement error and if 
we are capable - technically and economically — of measuring the phenomenon and making the 
required adjustment, the systematic error is determinable and should be eliminated. If the 
state of the technology or economics forbid a determination and subsequent elimination of the 
systematic error, it remains unknown and must be estimated. The following example is intended 
to illustrate the transitory nature of the limits between systematic and random errors of a 
measurement process. 


An unknown standard resistor with a nominal value of 1000 ohms was compared against a 
known standard resistor one day between 10:00 a. m. and 10:20 a. m. A ratio set was used for 
comparing the two resistors in air. The five values obtained were plotted on a chart. Figure 3. 
The ambient temperature was 25.4 degrees Celsius, varying by less than 0.1 degree during that 

time. The tem- 
perature coefficients 
of resistance of the 
two resistors were 
known, and all 
plotted values were 
corrected for a base 
temperature of 
25 degrees Celsius. 

The effect of 
temperature on the 

march 13 entire measuring 

system was not 
known. The 



Figure 3. Two Sets of Measurements of a 1000-ohm Standard Resistor 


ambient tempera- 
ture varied between 


approximately 25.5 
and 24. 5 degrees C 

at a typical rate of approximately 0. 5 degrees C per hour. It could, therefore, be assumed that 
the system was not at thermal equilbrium at any time; and the effect of this inequilibrium on the 
result of a single set of measurements would have to be estimated and a numerical value assign- 
ed to this effect as part of the systematic error if the value of the unknown resistor had to be 
determined on the basis of the first set of measurements. The nearly perfect agreement of the 
values obtained in the first set indicates that the resolution of the ratio set was insufficient for 


the measuring system to respond significantly to random fluctuations. An analysis of the random 
error of this set based on more than an inspection of the data, where four values are identical 
and only one differs from the other four by the smallest unit possible to resolve, would be of 
little value. 


The correct value of the standard resistor had been determined by a more accurate meas- 
urement process before and after the measurements described in this example and was 
1000 , 0120 ohms with a constant systematic error and with a random error of considerably less 
than 1 ppm. 


Between 2:00 and 3:00 p. m. of the same day, another set of measurements was made and is 
also plotted on the control chart. The ambient temperature was 24. 9 degrees Celsius. Again, 
the five measured values, corrected for the base temperature, agreed very well; perfectly, as a 
matter of fact. But they differed markedly from those obtained previously. It could be said that 
each set of measurements was under (simple) statistical control individually; but both sets 
together indicate an out-of-control condition, the two sets being offset from one another by a 
systematic error. Since no means were available to keep the ambient temperature more stable, 
and since the effect of temperature inequilibrium on the components of the measuring system 
was not known, this systematic error is unknown. If it had been possible to predict the system- 
atic error due to temperature difference and inequilibrium or to maintain the ambient tempera- 
ture more closely, the measured values could have been adjusted for the temperature inequili- 
brium, or the condition causing the inequilibrium could have been avoided. (The effect of 
temperature on the systematic error of the resistance measurement is given here as one 
example only. Many other sources of systematic error exist in practice and may influence the 
measurement uncertainty. ) 


The resistance of the unknown standard resistor was then measured repeatedly under 
similar circumstances. The results of these measurements were entered on the chart shown in 
Figure 4. Each plotted point is the mean of five successive determinations of the resistance of 
the unknown corrected, as far as possible, for 25 degrees Celsius. The points now are ran- 
domly distributed about a mean of 1000. 0124 ohms. Whereas each group of five resistance 
determinations was made with a very small random error and an appreciable systematic error, 
the new mean has a drastically reduced systematic error, but an appreciable random error. By 
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taking the measurements over a 
prolonged period of time, the 
systematic error of each group 
was allowed to vary and to 
become part, the major part in 
fact, of the random error of the 
measuring process. 


! Reference 1 describes the 

march i3 14 i6 phenomena of errors changing 

Figure 4. Twelve Means of Sets of Five Measurements their nature in the following 

terms: "If the cause system is 

@ Averages of values from Figure 4. enlarged, then what was pre- 

viously predictable bias may, 

in terms of the new cause system and process, vary in a random fashion and therefore be 
attributable now only partly, if at all, to bias. Furthermore, enlargement of the cause system 
requires re-evaluation not only of the bias involved in the stated accuracy but also of the stated 
precision. " 


Temperature changes, like many other causes of systematic errors, frequently occur in 
patterns, such as rough sinusoids or, in the case of closely controlled environments, in saw- 
tooth patterns, not randomly. The random appearance of the points on the chart of Figure 4 is 
due to the random selection of the times at which the measurements were made. It is interest- 
ing to note that an overestimate of the limits of uncertainty due to random errors would be 
obtained if the points would follow exactly a sinusoidal or a sawtooth pattern and the calculations 
were made as if they were normally distributed. In fact, all points of a sinusoid are within a 
region bounded by 0±\/2rj or within 1.41 standard deviations from the mean, and all points of a 
sawtooth curve of isosceles triangles with height h are within a region bounded by 

\ ± i J~3<r 


or within 1.73 standard deviations from the mean. 


The preceding examples illustrate, therefore, that: 

1. A measuring process ordinarily under statistical control will get out of control if the 
systematic error of the process varies, but control may be reestablished if enough 
changes are allowed to occur and the changes in systematic error are allowed to join 
the chance system of variations, 

2. If changes in systematic error are allowed to occur, the standard deviation of the 
measuring process will be larger than in the absence of such changes, and the sys- 
tematic error will be reduced. 

3. The time element will probably always change the "cause system" and transform con- 
stant errors to changing errors, and parts of systematic errors into random errors. 

The last example, however, raises another question. Is the 0.4-milliohm difference 
between the mean and the certified value attributable to a systematic error or could it be caused 
entirely by the fluctuations which a mean from a limited number of measurements could be 
expected to display ? Is there, in other words, a bias in the measuring process which should be 
eliminated ? Section V will show one way to answer this question. 


IV. ACCUMULATING SYSTEMATIC ERRORS 

The question now arises: Given the estimates for the maximum limits of all non-negligible 
systematic errors which must be taken into consideration, how is their effect on the measure- 
ment estimated? If the systematic error of a measurement is dependent upon several system- 
atic errors whose estimated maximum limits are given, how are they combined to yield the 
systematic error of the measurement ? 

For a general discussion of the treatment of systematic errors, let a system measuring 

the Quantity Q be described as Q = f , , >, where a, b, c. . . are components of the 

(,a, d, c , . . . ) 

system representing the known quantities A, B, C. . . The magnitudes of these systematic 
errors have been estimated to be not larger than e A , e^, e^,. . . , The propagation of error 

equation then gives the relative contribution of each systematic error term to the resulting 
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provided the errors are mutually independent. 

The application of the propagation of error equation (Ref. 5) is illustrated in the Appendix. 

Mindful of the facts that systematic errors are usually quoted as "±e." or ’M-e^ -e^"* * ,e * 

with a positive and a negative limit, and that they can be only positive or negative, but never 
both, metrologists have tended to devise methods to reduce the purely additive effect of the 
individual terms of equation 1. The most commonly used method to reduce this purely additive 
effect of errors is the rss-method by which the individual terms are squared and the square-root 
of the sum of the squares used as the estimate of the overall uncertainty. The conscientious 
metrologist will in the vast majority of cases avoid such an arbitrary reduction and adhere 
strictly to the additive form of the individual terms of that equation as written. 

Youden^) has shown that the combination of error terms in quadrature yields erroneous 
results in a chain of laboratories because it is deficient in logic. (See also Ref. 10 . ) In fact, 
any method to reduce the combined effects of several systematic error terms is arbitrary and 
can seldom be justified logically for the following reasons: 

1. Individual systematic error terms come from different populations, distributions, 
having different origins and means, and are unrelated. 

2. In most practical problems, the number of separate, individual error terms is small, 
and the probability that all error terms have the same sign is appreciable; four error 
terms still have a 12. 5 percent chance of ganging up with the same sign, and the con- 
sistent metrologist will not quote 3-sigma limits of uncertainty on the random error 
and take a 12. 5 percent chance on the systematic error. 

3. Although systematic errors are mostly believed to be overestimates, they may also 
be underestimates - and frequently are. 

But this problem appears to be more a theoretical than a practical one. In accordance 
with Juran’s principle of "the vital few and the trival many", in most practical cases only a few 
individual component errors determine the magnitude of the resulting overall systematic error, 
while the others are of little or no consequence. 

Thus remains to consider the theoretically possible but rare case of a large number, say 
ten or more, of systematic error terms of approximately the same magnitude where some 
mutual cancellation is likely to occur. In this case, the critical metrologist will accept any 
logical rationale to reduce the total error to something less than the sum of all individual terms. 
However, prudence and consistency would dictate that the resulting total error be not less than 
the sum of the nine largest individual terms when three-sigma limits are quoted for the random 
error and not less than the sum of the six largest individual terms when two-sigma limits are 
quoted for the random error. 

V. KEEPING MEASUREMENT ERRORS UNDER CONTROL 
IN A STANDARDS LABORATORY 

A standards laboratory involved in certifying standards for lower echelon laboratories 
must make considerable efforts to keep its errors under control. Ordinarily, it does not suffice 
that the measurement errors are determined periodically and that it is assumed the errors thus 
determined are typical and recurring at the same magnitude. A positive proof that the random 
error of a measurement was indeed typical and that no unusual systematic errors have occurred 
during the measurement would go a long way in enhancing the laboratory’s confidence in its 
measurements and contribute significantly to an eventual reduction of realistically quoted limits 
of uncertainty. 

Quality control practices, time proven and honored in production processes, provide us 
with excellent tools for the observation of the quality of measurements: Control Charts. 
"Measurement of some property of a thing is ordinarily a repeatable operation. This is cer- 
tainly the case for the types of measurement ordinarily met in the calibration of standards and 
instruments. It is instructive, therefore, to regard measurement as a production process, 
the ’product’ being the numbers, that is, the measurement processes in the laboratory with 
mass production processes in industry. "(4) Reference 2 gives detailed instructions on estab- 
lishment and maintenance of control charts. 



Figure 5 is a hypothetical example of a control chart for a 1000-ohm standard resistor. 
The resistor serves as our standard and is periodically calibrated against a higher echelon 
standard. Its value is established by a Trend Chart. Each time we calibrate a 1000-ohm 
standard resistor for somebody else, we also measure one of our own 1000-ohm standards (in 
this case the one whose control chart appears in Figure 5) by the same process and using the 
same instruments, in a pilot measurement. 



Figure 5. Control Chart of a 1000-ohm Standard Resistor 
(Measurements in Sets of Four) 


Each pilot measure- 
ment consists of a set of 
four measurements taken 
at different times. The 
mean of each set of four is 
plotted on the X-chart, and 
the range, the difference 
between the largest and the 
smallest measured values, 
is plotted on the R- chart. 
Upper and lower control 
limits are calculated as 
outlined in References 2 
and 7. 


Although occasionally a point may fall outside the control limits as a matter of pure 
chance, each time this happens the existence of some abnormality in the measurement is suspec- 
ted, and the measurement is repeated. If the point of the repeat measurement also falls outside 
the control limits, the existence of an abnormality is taken as being confirmed. The causes for 
this abnormality are then determined, removed, and the measurement is repeated. If the points 
on the X and the R-chart fall within the control limits, the measurement is considered valid. 
Since our customer’s unknown standard was measured by the same process, its value thus 
determined is also considered valid. 


Points of the X chart will reveal the existence of short term trends or cycles, and unusual 
systematic errors. The R-chart is predominantly an indicator of the quality of the measuring 
process, reflecting, among other things, the care of the operator and the control of the environ- 
ment in which the measurement was performed. Since a control chart is the result of a compil- 
ation of a considerable amount of data concerning one measuring process, the average 
experienced range R can be used to determine the standard deviation of the measuring process. 
’’Most experimental scientists have very good knowledge of the variability of their measure- 
ments, but hesitate to assume known a without additional justification. Control charts can be 
used to provide the justification. "( 7 ) 

The last point on the control chart for averages and that for ranges in Figure 5 represent 
the last calibration of 1000 ohm resistors performed by our lab. The average of the four meas- 
urements taken on the pilot resistor charted in Figure 5 was X= 1000. 0121 ohms and the range 
of the four measurements was then 3. 8 milliohms. The recent grand average of pilot measure- 
ments made with this resistor isX= 1000. 0123 ohms and the average range is R= 2. 2 milliohms 
as plotted. Upper and lower control limits on the X-chart of the averages are at a distance of 

A 2 R = 0. 729 x 2. 2 = 1.6 milliohms 

from the grand average X. Upper and lower control limits of the R-chart of ranges are at 
D 4 R = 2. 282 x 2. 2 = 5. 0 milliohms 

and 

DgR = 0 x 2. 2 = 0. 0 milliohms 

respectively, as plotted. The estimated mean standard deviation of the calibration process 
which yielded these results is 

6 = f^ = 2^9 = 1,07 milliohms ’ ( 2 ) 

and the uncertainty of measured value of 1000.0121 ohms due to random error at 3-sigma levels 
is _ 

3S X= 1.6 milliohms 

The factors Ag, d^, D^, and are tabulated in references 2 and 7. 


6. Trend Chart of Recent Values of Resistor A 


The two last 
points, like the 
others before, fell 
within the control 
limits, indicating 
that no abnormal 
systematic or ran- 
dom error had 
occurred during 
the measurement 
and that the meas- 
urement was , 
therefore, typical 
for this particular 
process. 


The distribution of the points in Figures 3 and 4 is not necessarily typical for normal 
laboratory comparisons between standard resistors where it is frequently possible to control 
the factors influencing the measurement (e.g. temperature) more closely and attain narrower 
uncertainty bands, i. e. narrower regions bounded by control limits. 

The difference between the value of the standard as determined by the Trend Chart and the 
grand average X on the Control Chart must be considered a systematic error of known magni- 
tude and sign if the difference is statistically significant. A significant difference must be 
removed by adjustment or by applying this difference as a correction to all measurements made 
by the process under consideration. 

The difference is statistically different at a level a, if it is larger than 


where z^ is the standard normal variable or the ordinate of the normal curve for a cumulative 
area of p under the normal curve, and where p = 1 - ^ . 


Using Equation (3), we can now answer the question at the end of Section III C. Let us 
assume that the average range of the five measurements (n=5), was R = 0.7 milliohms. Choos- 
ing a = 5% or 0. 05, p = 0. 975. We first calculate ft- from Equation (2) as 

miUlohms - 

Then, from Equation (3). 

u= 1.96 0.26 milliohms 

n/5 

Since the difference between our measured average and the certified value is 0.4 milli- 
ohms and, therefore, larger than m, the difference is significant, and we conclude that a bias 
exists which calls for immediate correction. 


To allow for changes in the systematic error contributed by the measuring process or the 
standard, the grand average must be updated periodically, at which time it may be necessary to 
omit some of the oldest points and recalculate the average on_the basis of the latest points. 
Between 10 and 25 points should be available for calculating X and R initially, and their values 
should be updated when additional groups of points become available. 


VI. CONCLUSION 

The majority of all sources of uncertainty which a metrologists normally encounters must 
be treated as systematic, even though many of them may have originated as random errors. The 
boundaries between systematic and random errors are fluctuating, and systematic errors may 
at times recover their random nature if we increase their ’’cause system”, i. e. frequently the 
time period over which we consider them. The statistical treatments developed for the deter- 
mination and analyses of random errors do not normally apply to systematic errors. We have 
seen how we can determine the magnitudes of systematic errors of standards and measurements, 
how we can control them, and how a control of the systematic errors, combined with a control 
of the random errors of measurements, can lead to a, positive and definitive control of the 



entire measurement process. Trend Charts for standards and Control Charts for measurement 
processes are invaluable tools for the sophisticated metrologist to enhance his knowledge of the 
behavior and uncertainty of his standards and measurement processes. 

The emphasis is on collecting and analyzing information. Proper handling of data can then 
help to convert available information into a different form, but data manipulation cannot be a 
substitute for knowledge. 
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APPENDIX 


Example on Accumulating Systematic Errors 

This example illustrates the calculation of the systematic error incurred in measuring the 
value of a 1 gigaohm resistor (X) against a 100 kiloohm standard resistor (A) whose value was 
determined earlier with a total uncertainty of 0. 018%. The apparatus used in the measurement 
is a differential high resistance bridge shown in Figure A. 



CALIBRATED DECADE 
RESISTOR 


UNCALIBRATED 
VARIABLE RESISTOR 


Figure A. Differential High Resistance Bridge 


Hence, 

x = V B 2 , N). 


This bridge is first balanced with 
resistor X connected to terminals 1 and 
2 and with the calibrated decade resistor 
set to zero (Bi=0). Resistor X is then 
disconnected and the bridge balanced 
again by increasing the resistance of arm 
B by an amount B2, using the decade 
resistor. The measured value of X is 
then 



The values and uncertainties of the individual elements of Equation (A) are found as tabulated 
in Table I. 


Table I. Values of Elements, Equation (A) 


Element 

Measured Value 

Total Uncertainty of Measured Value 

Symbol in % Magnitude 

A 

100 005 S2 

e A 

0.018% 

18 Q 

B 2 

9.9005 n 

e B 

0. 27% 

0.027 Q 

N 

1.000 

e N 

0.011% 

0.000 11 


Applying Equation (1) where Q now must be replaced by X of Equation (A), the systematic 
error of the resistance measurement due to the uncertainty of the bridge elements then becomes: 
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ax 
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(B) 


The partial derivatives of X with respect to A, B2, and N are then respectively, 
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so that (B) becomes 
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9 

The value of X, using Equation (A), calculates to be 1. 009 6 x 10 ohms, so that the limits of 
uncertainty due to systematic error of the bridge circuit and the standard are at ± 0. 32%. 
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QUALITY “ A TRUE TEST OF MANAGEMENT 


Harry J. Lessig, Consultant 
R„R. 4, Oakdale Drive 
Springfield, Illinois 62707 


Over the past several decades, many tests have been devised for determining the 
effectivity of various levels of corporate managers; however, in the final analysis, 
the unit of measurement most frequently used has been the dollar on the bottom line of 
the profit and loss statement. As a matter of fact, with few exceptions, the dollar 
remains as the basic unit of measurement of managers at all levels, including managers 
of Quality and Reliability. I take, no issue with this; hox^ever, the effect of poor 
quality frequently does not show up on the bottom line immediately and managers may not 
realize the ultimate effect of poor quality until it is too late. 

Although every industrial manager plays a definable role in the quality or goods 
produced, and despite the fact that it is the ma in part of every quality manager’s job, 
the key individual in producing high quality products is unquestionably the General 
Manager. Without his commitment to product quality, and his support when the going 
gets rough, even the most professional Quality Control Manager can do very little in 
preventing the manufacture and shipment of substandard products. If, on the surface 
this seems to sound strange, let us probe more deeply. 

It has been widely said that "quality is everybody's job" and, conversely, it has 
been widely demonstrated that "everybody’s job is nobody’s job”; therefore the buck 
must stop somewhere. 

Engineering must design a product which can be produced economically and which 
will satisfy the customer’s expectations; Manufacturing must maintain the design in 
fabricating the product; Quality and Reliability must assure that the integrity of the 
design has been maintained; and Marketing must sell the product only for those appli- 
cations which will be satisfied by the designed performance. Therefore, if a part of 
everybody's job influences product quality to varying degrees, someone must coordinate 
and put it all together - The General Manager, The buck stops there. 

I have been intimately acquainted with the professional performance of many qual- 
ity managers; yet, in my opinion, rarely does one exist who is not capable of doing a 
much better job than he is permitted to do, even within the fiscal restraints of his 
budget. Equally as rare is the general manager who does not loudly proclaim his dedi- 
cation to product quality; however, if the past performance of general managers had 
matched their verbal protestations, there is some reason to believe that acronyms such 
as CPSC, OSHA, DOT, and EPA might be substantially less prolific. 

History has taught many lessons regarding product quality, but little seems to 
have been learned. In good times and bad times alike, there has always been a great 
demand for products with a reputation for high quality. In the depression of the 1930 's 
there was a substantial demand for Cadillac motor cars, just as there was in the recent 
sales downturn of 1975. What rebate was offered on the purchase of a Cadillac in 1975 
when others were offering $400 to all takers if they would buy brand "X"? 

Quality is easily recognized and sells with little huckstering. Despite the 
fervent wishes of many of those on the commercial scene today, consumer awareness is 
not going to decrease; rather, it is increasing rapidly. The well known television 
commercial which advises, "Promise her anything, but give her Arpege" may very well 
sell perfume; but a manufacturer who takes such advice literally concerning his product, 
will soon learn of his error, and perhaps disastrously. All too recently, consumers 
all over America demonstrated that they couldn’t be promised a higher priced automobile 
and then be delivered a car with a lower priced engine. Today’s consumer will be satis- 
fied with nothing less than a product which fully lives up to his expectations, and he 
becomes quite visible and vocal when a product falls short of its advertised perf ormance . 
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The author has been a quality professional for over 40 years, and honesty, inte- 
grity, and high ethical standards of conduct have always seemed very important to him. 
Unfortunately, some of the supervisors to whom he reported did not entirely share the 
same degree of value he placed on those elements of stewardship. Blessed is the 
General Manager who has learned the value of high product quality, and doubly blessed 
is the Quality Manager who has such a General Manager. 

In many instances the author has enjoyed some excellent general managers who not 
only permitted him to do his job, but also gave him every bit of encouragement and 
support he could ask for; yet, too many previous supervisors seemed to lack the talent 
to lead a manufacturing team through the pitfalls of production difficulties. Instead 
of taking the lead in solving production problems, they could only insist on the accep- 
tance and shipment of shoddy products that did not measure up to the designed, adver- 
tised and certified standards. 

It might be understandable if the author was a rigid conformist, who unyieldingly 
insisted that parts were not acceptable if there was even the slightest variation from 
blueprint dimensions. Rather, he has always maintained a pragmatic approach to design 
deviations and, if conservative engineering judgement indicated "fitness for use", 
there was no question as to acceptance. Use the parts but adjust the process so that 
future production would comply! 

Practically every Quality Manager is able to cite varying numbers of traumatic 
experiences wherein he has agonized over the anticipated effect of having his profes- 
sional judgement overruled by his supervisor. 

It is with substantial concern that the author has noticed what seems to be a 
significant downward trend in individual ethical performance, not only at the General 
Manager level, but at all levels of management. The maintenance of a high standard of 
ethical performance is never the easiest course of action, and rarely the most pleasant 
one. Worshipping at the shrine of expediency has reached endemic proportions, and the 
self-discipline necessary for doing the unpleasant seems to be a steadily vanishing 
characteristic in all walks of life. 

Each of the professional, engineering, and technical societies has an excellent 
Code of Ethics which has been refined over the years so as to embody only the highest 
ideals of integrity, morality, and stewardship. The American Society for Quality 
Control is no exception. 

Everyone fully subscribes to these Codes of Ethics and the principles they reflect 
however, in too many instances, there seems to be an inadequate effort to apply them 
to the difficult business decisions every manager is called upon to make each day. 

Today's colleges and universities have done very little to include ethics educa- 
tion into their professional and business curricula and, unless there is a concerted 
effort to make students more aware of the conflicts they will inevitably encounter in 
their professional careers, sufficient enthusiasm will not be generated for establish- 
ing ethics courses. 

When incoming inspection of purchased items reveals that substandard quality has 
been received, it has become almost automatic to assume that the supplier has an inade- 
quate quality control system which does not properly catch defective products. Addi- 
tionally, the notification of rejection will also ask what the supplier will do to 
improve his quality system so as to preclude a recurrence. Perhaps we should not be 
too hasty in making such assumptions. 

One cannot help but wonder how many of these instances, if the truth were known, 
were not caused by the failure of the supplier's quality system to identify the unsat- 
isfactory product but rather because the General Manager overruled the rejection by 
his Quality Manager and ordered the material to be shipped regardless of the known 
unsatisfactory condition. It has happened frequently, and will continue to happen as 
long as it goes unrecognized, ignored and tolerated. 

When a professional Quality Manager must operate in such an environment, one of 
three things must happen: 1. The Quality Manager must develop an infinitely elastic 
aixd convenient conscience; 2. He must attempt to chart a most difficult if not impos- 
sible course of ethical conduct which will permit him to retain his employment, and 
thereby accept the potential risk of violating his professional code of ethics; or 3. 
Seek another less frustrating environment. 


In either event, considering the tremendous hazard of legal liability which hangs 
as the Sword of Damocles over the head of every Quality Manager, it seems most judic- 
ious that he maintain a bound ledger or log book concerning those cases in which his 
professional judgement was overruled. Each item should be dated and give a full 
account of all particulars of the matter including the item involved, the quantity, a 
description of the product quality deviations, the degree to which it was present, the 
name of the individual who directed acceptance and/or shipment of the items which were 
known to be defective, and any other pertinent data having a bearing on the matter. 

Now in order to preserve some semblance of fairness, the question of overriding 
the Quality Manager's decisions should be examined from the viewpoint of the General 
Manager . 

It seems quite reasonable to assume that no one, even Quality Managers are so 
infallable that their decisions are invariably correct. Additionally, in some circum- 
stances the General Manager may have pertinent information which is not available to 
the Quality Manager and, in rare instances, information so confidential that it may 
not be revealed. In those instances, the General Manager's veto action may very well 
be entirely justifiable; however, if such instances occur somewhat regularly, it: seems 
reasonable to conclude that either the Quality Manager is somewhat less than optimally 
effective, or the outgoing product quality level is leaving much to be desired. 

No manager's job is an easy one, but quality is_ a true test of management. 
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WHY CALIBRATE OURSELVES TO DEATH? 


Jack F. Wilcox, 
Associate Division Manager 
Quality Assurance 
Bechtel Power Corporation 
Los Angeles, California 


For many years we've been involved in all kinds of calibration programs. They're 
not only required by various regulating documents; why, even common sense demands 
them. 

This presentation isn't made to demean these efforts. Rather, my complaint is 
directed toward the overwhelming complexity and costs of programs based primarily on 
periodic recalibration. Those costs might be justified if they honestly satisfied 
our single primary objective; namely, acceptance of only those products meeting their 
design criteria. Too often, the objective is missed regardless of the efforts ex- 
pended: We initiate elaborate procedures to detail the turn of each button or handle 
for every measuring instrument we use, be it electrical, mechanical, hydraulics, or 
what have you. Then we set up time cycles for re-verification of those characteristics. 
Next we initiate "trend charts" to identify needs for shortening the cycle and to 
identify possibilities of cycle lengthening where appropriate. The fallacy is that all 
these exercises are attuned, not to the direct acceptance of a quality product, but to 
statistically arrive at a magic span of time within which an instrument will probably 
remain "in tune". I say probably, simply because the cycle ignores the damages caused 
by an instrument being dropped on a concrete floor or getting caught in a machine. 

We certainly want to minimize these damages. 

To begin, let's "Value Engineer" the task for a minute: If the calibration 
program we propose actually validates the authenticity of our product acceptance 
examinations, then we've accomplished our basic objective. No matter how bare, no 
matter how frugal it is, the program has to be recognized. Maybe it doesn't meet all 
the little tricks, controls, and niceties of some particular specification; but then 
if we plan a little deeper, maybe it can. There's no magic involved; let's just 
examine the possibilities of simplification by modifying applications of the term 
"periodic". All the Standards I know read something like. "Shall be calibrated at 
specified maximum periods of time, ojr use ". 

Summarily, my proposal is to assign, wherever feasible, to the instrument user, 
the inspector, the machinist, the technician, complete responsibility for checking 
the accuracy of the instruments he uses. This checking is done to "work", "reference", 
"secondary" standards, or whatever you want to call them, and these standards then, 
instead of the instruments, are periodically checked in the normal fashion. 

My first exposure to an honest, direct, and effective calibration program was 
back during World War II at a gear manufacturing organization. The company fabricated 
a high percentage of precision aircraft gears used by Wright Aeronautical and Pratt 
and Whitney. Both machine operators and inspectors used their own personal standard 
mechanical measuring instruments for most verifications. Each time a machinist set 
up to run a particular operation, he took his instruments to a gage crib (sometimes 
the day before) with the process drawing of the dimensions he was to generate. The 
tools were then set to those exact dimensions and paper slips were initiated identi- 
fying the part number, operator number, dimensions, date and the gage inspector who 
did the setting. Likewise with the bench inspector who checked each individual part 
prior to the next machining operation. Both operators and inspectors were required 
to maintain the calibration slips in plain view at their work site. 

The Cage Inspectors literally worked their "Masters" off; but during five years 
as Quality Engineer with the company, I kmw of no recorded incidents of any serious 
deficiencies traceable to inadequate calibration control. Oh, we had the usual 
problems, like a surplus of discrepant "set-up" pieces, breakdowns of heat treat 
cycles, too much or too little grind stock, sloppy fixtures and the like, and we had 
some run of human problems too. 
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An Air Force Inspector stopped at an inspection station where a young lady 
inspector sat, surrounded by boxes and skid-loads of gears. She was using a Rockwell 
Hardness Testing Machine to test core hardness. She carefully wiped each gear, placed 
it on the anvil, flipped the lever, waited until the handle stopped, released the 
pressure and removed the gear. The Air Force representative asked, "And what is your 
job here young lady? It looks very important". The young lady took a gear out of 
the tester and turning on her stool, handed it to the representative: "You're darned 
right it's important. See that little bitty hole? I have to put a hole like that in 
every single one of all these gears before we ship them". 

Anyway, after exposure to 100% calibration, I moved to the Ford Motor Company, 
Aircraft Engine Division, to build Pratt and Whitney Aircraft Engines. The calibra- 
tion program here was entirely different, with all M and TE (measuring and test equip- 
ment) on periodic cycles varying from each work shift for some to annual on others. 
Funny, how much faster the "GO" members always seemed to wear out! 

The total number of different micrometers as a single equipment item, reached 
into the thousands. They were controlled by what was then known as the "Three-Card" 
System. Card Number 1, filed in a Master Gage Lab, identified an inventory number 
assigned after receiving inspection acceptance, and carried the plant work area crib 
number to which the instrument was assigned. Card Numbers 2 and 3 were kept in the 
Work area cribs with the instrument. Card Number 2 recorded, along with the inventory 
number, the model, etc., and wear tolerance, results of inspections, inspector identi- 
fication, and the next due date. It was filed by inventory number. Card Number 3 
containing identical information, was maintained in a "calendar tickler" file set up 
by month, week, and day. The Number 3 or last card, then, identified the work load 
for the area gage inspectors for upcoming days, weeks and months. 

I'm going through these details for a purpose - so that you can visualize the 
enormity of labor connected with such an operation, not only with actual calibration, 
but also with the clerical work of transferring card information. The plant had some 
twenty-seven area gage cribs, each manned by several gage inspectors and clerical 
support, 

I really can't take credit for the next innovation that took place. Someone 
decided that checking of the standard instruments like micrometers could be more 
expeditiously checked right in the manufacturing work areas. The gage inspectors lost 
considerable time searching for people with micrometers signed out. The main reason 
for this was that machinists and inspectors preferred to keep the instruments locked 
up rather than check them in and out every day. In preparation for moving the gage 
inspectors out "to the action", we built some standard machined rings, pyramids, etc., 
to be used as work standards when mounted on push- wagons. Had I been in the wagon 
manufacturing business, I might have been successful; but since I wasn't, (who ever 
heard of a successful inspector?), I proposed that we forget about using them. Instead, 
and possibly to avoid responsiblity for scrapping hundreds of fabricated "work masters", 
I had the fixtures mounted outside each gage crib window on a supporting shelf and hung 
a sign over them. The sign announced a new rule: "Users are responsible for their 
own instruments. Check your mike before you use it". The procedure dulled some of 
the "luster" from the lofty gage inspector jobs and turned them into part-time 
instrument repairmen. But, we were able to detroy the Number 3 cards for all micro- 
meters, reducing the "worklog" volume by over 50%. Individual machine operators and 
inspectors didn't mind making their own checks but some gave the gage men fits because 
their personal 'feel' on a particular instrument was different than as adjusted by 
the last setting. 

My "User" calibration program permitted a considerable reduction in workforce. 

The reduction turned out to be just a "token"--the entire plant closed down. 

For eleven subsequent years, I worked for a company who had a much more elaborate 
and modern calibration program. Everything was on periodic cycles but IBM took care 
of all the bookkeeping. They hired one "Planner" for every four gage inspectors. The 
Planner initiated special check-sheet instructions for each and every type and size of 
measuring instrument. These check-sheets had to be meticulously and faithfully used 
by the gage inspectors, with all individual accepted characteristics stamped off 
exactly as if the employees were documenting a product acceptance rather than cali- 
brating a standard measuring instrument. 



Extremely advanced methods and equipment were installed including GE Electronic 
Gages to calibrate gage blocks. We had a few problems: We found that a female gage 
specialist needed approximately one hour to check a single Hoke block. This worked 
out to a cost of from 50 to 60 man-hours, plus a 133% burden rate for calibration of 
a 50-piece block set. We went back to sending the Gage blocks to the Naval Ordnance 
at Pomona. Then three more specialists were hired to "Trend" various wear patterns 
of different types of gages. The trends told us many things: Go, No-Go Plug Gage "A" 
at .9995/1.0000 didn't wear out for a long time, so the recheck cycle was extended 
from weekly to every two months. Problem--when calibration of all "A" plug gages 
went to two monthly checks as a result of the "trend" study, we got into some assembly 
problems because of undersized holes. Reason — while some of our "A" plug gages 
lasted for up to eight months, we had other applications of "A" plugs that wore out 
in about three weeks. 

But progress will not be denied. The gage calibration program continued to 
expand and refine its procedures. In all honesty, we never had to spend as much 
money reinspecting hardware as we spent on the calibration program--that is if we 
didn't include the product rework/repair costs. Following are some possibly surprising 
facts: Inspectors very seldom bothered to check their instruments --why should they? 

Somebody else was responsible, not they. And as far as the gage inspectors were con- 
cerned, why fight City Hall? Work loads were pre-established so a micrometer got 
checked when the IBM said so and that's all. When damaged or worn instruments were 
discovered, so what? The "Trends" would take care of it and close up the period 
cycle. What's that old adage about "The best intents of men geng aft aglae?" 

Some years later, up in Richland, Washington, I ran into a situation that really 
provided an opportunity to try my "category" idea. The Hanford complex was com- 
prised of a family of AEC contractors, serviced by a small but top-notch calibration 
laboratory working under the auspices of one of the contractors, Batelle Northwest, 
who was also my employer. The trouble was the contractors didn't make much use of 
the lab, but rather tended to avoid it. 

There were several thousand items of M and TE on several recorded contractor and 
divisional inventories. When an item was sent to the lab for calibration, it x?as 
placed on a period cycle. When the cycle was up, a notice was sent to the owning 
contractor division or department requesting submission for a re-check. The notices 
were almost always ignored. Can you imagine the stack of paperwork, notices, trans- 
mittals, second and third requests? The calibration lab barely managed to stay open. 
Its operating cost had to be charged direct, that is, to the department and contractor 
for whom the work was done; so - no work, no pay - very few people. 

In defense of the contractor departments and scientists, they had their own 
reasons for ignoring the recalls. Some of them were reasonable. A small group of 
scientists working on say a six-month $240 K experiment pulled out maybe 30/40 M and 
TE items from their division stock of possibly 800 to 900 items. They would send 
possibly 4 or 5 particular gages to the lab prior to making their test set-ups, but 
only if they thought their accuracies might be suspected to the extent of degrading 
their generated data. They didn't manufacture a product, they ran experiments and 
published results. Often they rightfully felt that so-called "built-in" accuracies 
were good enough for their needs. They would use a "set-up" of instruments continuous- 
ly on a particular experiment for the six months and then maybe not again for a year 
or two. They bought their own equipment as chargeable to this or that experiment and 
the purchase included a supplier certification which was filed for reference over the 
years. So long as they kept the instruments to themselves, they could use it as they 
wished, but when they sent it to the gage lab for calibration, or repair, it automa- 
tically appeared on the recall IBM and was forever carried as delinquent while in 
truth it might have been out of use for months or even years. 

My helper this time was the AEC; they demanded that Batelle up-grade their 
program. I officially initiated a "category” calibration program like this: First 
of all I decided that we only needed to identify and control gages actually being 
used, so we documented the fact that gages without labels would not be used in any 
experiment or machine work since they would be considered as strictly "in storage". 










Strange as it may seem, the Richland, Washington, office of the AEC audited our 
new program and praised it - but not the first time. Remember - the backbone of the 
program is that the "user" must check his A's prior to use. If I ask a man working 
on an experiment, "Row did you check that instrument?" and "What B reference instru- 
ment did you use?", the person must be able to (1) produce the B referenced standard 
and (2) demonstrate proficiency necessary to adequately make and evaluate the compari- 
son. 


On the AEC ' s first Batelle audit, several departments, including the machine 
shop, came through with flying colors; but on one little project experiment, a 
working scientist had put A stickers on every instrument including a set of blocks. 

His reasoning: "I bought the gages myself, for my own work and I have supplier 
warranties. I believe those warranties equate our gage lab checks." He passed OK 
on the next audit. 

The Argonne Laboratory, as an organization, was a little like Batelle Northwest, 
but operated under the auspices of the University of Chicago on contracts for the 
AEC and more recently, the Energy Research Development or ERDA organization. 

A preponderance of M and TE were on periodic cycle cards for calibration and on 
IBM runs for inventory accountability, but the IBM didn't carry calibration status. 

In September of 1972, two file cabinets were full of calibration cards repre- 
senting instruments, some of which hadn't been seen by the single assigned gage 
inspector for as long as four and five years. The Quality Control Supervisor, 
Inspection Foreman and I spent one weekend removing all labels from all instruments in 
the four working inspection areas - that is, from all inspection equipment. Two days 
later, three ERDA auditors appeared and most fortunately for me, one of them was a 
participant from the Richland, Washington, office who had reviewed and was familiar 
with the Batelle program. 

We presented a single-page document, identifying a categorized calibration 
program basing primary responsibility upon the user supported, by appropriate 
reference masters under periodic cycles. The document was accepted with a 60 day 
commitment for inspection. 

Announcements were posted describing the program, explaining the use 
responsibility concept: 


NOTICE 


To: All Inspection Personnel 

From: J. F. Wilcox 

Periodic recalibration of micrometers and certain other common hand measuring 
instruments is hereby discontinued. 

The accuracy of these instruments shall be the responsibility of each using 
inspector. He shall check his instrument to a suitable reference standard each time 
it is withdrawn from bench, drawer, cabinet or crib. Should anything be found wrong 
with the instrument, it shall be presented to the inspection supervisor for repair, 
re-set and/or replacement. 

Quality Control will continue to provide requested calibration or repair of 
such instruments for production personnel. 

Additional reference standards will be supplied to various shop areas in the near 
future for user convenience. The reference standards will be incorporated into 
regular periodic recalibration cycles. 



Categories were defined as follows: 


CATEGORY A 


USER MUST CALIBRATE 

A THIS INSTRUMENTS 
SUITABLE REFERENCE 
STANOAROPRIORTO USE I 
ANl 272(3/73) | 


USER MUST CALIBRATE 

A THIS INSTRUMENT TO 
SUITABLE COMPARISON 
STANDARD PRIOR TO USE 
ANL-27213/73) 


Category A - Instrument which must be examined and calibrated by the user to suitable 
reference standards each time they are withdrawn, for use, from a tool crib, locker, 
desk, or other place of storage. These instruments are physically identified by the 
ANL Quality Assurance White Calibration Label (Form No. ANL-272), assigning such 
calibration responsibility to the user. Instruments found discrepant shall have 
identification removed and shall be submitted for repair prior to re-identification 
as Category A. 


CATECORY B 


A THIS INSTRUMENT DUE 

FOR NEXT CALIBRATION ON 


A DUE FOR CALIBRATION 




ANL 273(3/73 1 


ANL-273 (3/73) 


Category B - Instruments which require regular examination and calibration to refer- 
ences by other than the user. This category generally includes reference "standards" 
used for calibration of Category A instruments and also instruments requiring special 
setups/environments, arrangements, or equipment to accomplish the examination. These 
instruments are physically identified by the ANL Quality Assurance yellow calibration 
label (Form No. ANL-273). Label indicates calibration expiration date, serial number 
of instrument and name of technician. 

NOTE: Accuracy of reference standard used imist be traceable to U.S. National 
Bureau of Standards. 


Certain entities not formally covered in the original Batelle program were 
addressed in gradual refining of the Argonne program. For example, the following 
direction dealt with items like V blocks, parallel bars, square and cube blocks: 

"Assist or support tools must be visually inspected for any burrs, damaged edges, 
or other surface irregularities that could affect inspection results. All such 
irregularities must be removed by stoning or other appropriate means prior to use". 

Category A instruments such as micrometers were required to be checked in four 
rotational thimble positions using set-masters, prior to use. 

Category B instruments were calibrated to documented descriptive procedures, 
initiated for general classes or types to comply with the ERDA imposed Standard RDT 
Calibration F 3-2T. 

During several years of continued operation, a few refinements, clarifications, 
and such have been incorporated, mainly as ideas submitted by the people themselves. 

Those contributions tend to illustrate generated employee enthusiasm and their 
pride in fulfilling of regular calibration responsibilities. And, I truly believe 
that such pride is important to a successful Quality program, don't you? 
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PROCUREMENT QUALITY PLANNING AND CONTROL 


William G. Gage Jr. 

Senior Procurement Quality Assurance Engineer 
General Dynamics - Fort Worth Division 
Fort Worth, Texas 


A successful Procurement Quality Planning and Control program must start out as a 
plan itself. A planned Quality Assurance - Procurement team effort must meet with all 
potential sub-contractors and suppliers on all levels of management and during the 
various phases of development and production to define and refine the planning and 
controls necessary to any project undertaking. 

To digress momentarily, we must define several very critical concepts: First, 
there should be a written procurement quality policy which states "This company will 
deal only with sub -contractors and suppliers who are: 

(a) Capable and well qualified, 

(b) Understand the requirements, 

(c) Are willing to adhere to the requirements" 

Second, Quality Assurance functions are NOT responsible for Supplier quality, Procurement 
specifically the Buyer or Purchasing Agent, is responsible for quality; Quality is a 
balance of; 

(a) Compliance to Technical requirements 

(b) Competitive pricing 

(c) On time delivery 

Third, Procurement must take the lead in strengthening Supplier performance: 

(a) Responsibility for Supplier Quality lies with the Supplier, 
his personnel are responsible and his management must make 
it mandatory, 

(b) "True -Cost-Of -Purchase is the sum of all costs including; 

receiving, inspection, testing, rejecting, returning, plus 
Re-receiving, Re-inspecting, Re-testing and other like costs 
Fourth, Procurement must assure the Supplier's under standing of their responsibilities, 
which include: 

(a) Planning for the total satisfaction of contract requirements 

(b) The burden of proof for Quality lies with the Supplier, 

(c) Certification and delivery of acceptable quality levels, 

(d) Quality conformance is submitting only items that meet all 
the requirements, 

(e) "Running his own Show", no Supplier should sit back and 
expect the Buyer or Buyer personnel to give him a push or 
run it for him. 

With all these items you may be wondering, "What is the Procurement Quality Assur- 
ance role?" Procurement Quality Assurance is needed to: 

(a) Assure that the technical and contractual requirements are 
properly defined in the request for proposal, 

(b) Participate in the evaluation and selection of qualified 
suppliers, 

(c) Help suppliers understand requirements, 

(d) Evaluate compliance to requirements, 

(e) Identify causes of variability and act to eliminate them, 

(f) Participate in non-conforming materials control and disposi- 
tion, 

(g) Rate supplier performance and conformance. 

Every Procurement Quality Assurance action is aimed at the primary goals of delivering 
a product or service that meets the requirements fully, on schedule and for the quoted 
price, which provides a fair profit for the supplier. 

Now, back to the planning and control function. Procurement Quality Assurance 
planning must be a carefully integrated portion of the entire procurement cycle. It 
must mesh with and support the overall objectives, it must be results oriented, it must 
be systematic and it must be cost effective. A” planner, be it a Quality Engineer, Quality 
Manager, or Quality Analyst, must carefully and methodically think through the require- 
© 1978 American Society for Quality Control 



raents and develop a step-by-step program which will assure that the final plan will 
contain all the necessary functions to meet the requirements and objectives. The 
planner must assure that all the alternatives have been considered and the best for the 
circumstances selected. An important part of planning is the consideration of contin- 
gency factors, the "What If" items, those items which fall into this catagory which 
prove to be significantly important should have a contingency plan developed and ready 
for use, should the need arise. 

There are several steps which are vitally important to a Procurement Quality Plan: 

(1) Requirements Definition ( 7 ) Source Appraisel 

(2) Supplier Evaluation (8) Corrective Action Program 

( 3 ) Supplier Selection ( 9 ) Failure Investigation 

(A) Pre and Post-Award Conferences (lO) Source Inspection and Test 

( 5 ) Product Quality Plan ( 11 ) Receiving Inspection Plan 

(6) Surveillance and Audits (l2) Supplier Performance Ratings 

Requirements Definition- Defining the requirements makes a close relationship with 
Program Management and Design Function personnel mandatory. The relationship must be 
built upon mutual respect for each others abilities, an understanding of each others 
functional responsibilities and the realization that the decisions of each has a pro- 
found effect upon the project results. All requirements must be spelled out in suff- 
icient detail to eliminate any doubt but at the same time must be kept concise and brief. 
Requirements must follow a standard format and include all aspects of equipment, testing, 
and qualification specifications. 

Supplier Evaluation- All responding suppliers will require on evaluation of some 
type. Possibly it will consist only of the review of the proposal response, but if it 
is one of the finalists (especially on large contracts) a qualification' survey and 
on-site review of the facilities and capabilities will be made. Additionally, a review 
of past history and supplier performance ratings should be made. The performance of a 
survey is to assure that the management systems employed provide the proper management 
and procedural controls to assure adherence to contractual requirements. A review of 
past history provides insight into the implementation and results of the system appli- 
cation . 

Supplier Selection- The supplier finally selected will be the results of the 
Procurement-Engineering-Quality evaluation of Technical, Pricing, Contractual responses 
plus Capability Ratings. The selection will be a blending of the Technical requirements, 
Price, Schedule and Quality that satisfies the Buyer's needs. 

Pre and Post-Award Conferences- These conferences are hard to define, some companies 
have both and they are formalized, while other companies have neither or they are very 
informal and then there are all combinations in between. To avoid a waste of money and 
time, if a conference is necessary, it should be formalized at least to the point of 
having an agenda, to assure that all points are covered. This agenda shouldhavo inputs 
from allfunctions of boththe Supplier and the Buyer. There should, also, be a set of 
minutes published setting forth the discussions and their results and ail agreements and 
interpretations recorded for future reference. Pre-Award conferences are particularly 
sensitive situations and must be handled carefully. Each participant must be given 
exactly the same information about requirements or other factors to prevent any poss- 
ibility of giving an unfair competitive advantage to one participant or another. Also, 
extreme care must be exercised to prevent revealation of confidential information being 
communicated via a discussion which would expose proprietary information of another 
participant. Post-Award Conferences should use carefully prepared presentations which 
set forth and re-iterate the agreed upon goals and objectives. In all conferences, 
each item that comes up for discussion should be discussed forthrightly and openly, 
there should be no"beating-around-the-bush", "weasel-wording" or "double-talk", if 
something needs saying it should be stated plainly, with a spirit of clearing up the 
facts, so as not to lead to misunderstanding or mistrust at a future time. 

The Product Quality Plan- This plan is the connecting link between the Total 
Quality System and a specific product. It is the application of the Quality System 
to the manufacturing processes which produce the characteristics of the product. The 
plan must identify the major tasks, how the tasks will be controlled and a schedule 
for accomplishment. A flow diagram must be created which identifies the routing of the 
product through the conversion process, origination of critical characteristics, the 
inspection and test points, along with references to procedures, instructions and test 
plans . 


Surveillance and Audits- Surveys and audits are used to evaluate the implementation 
of the Quality System and the Product Quality Plan- Since it is impractical and un- 
economicalto duplicate inspections and tests performed by the Supplier, one of the 
better methods in use is the surveillance of the overall operations and audit of "key 
points" in the hardware or process cycle for results. Review and evaluation - of Quality 
procedures, instructions and data for proper application and interpretation. Hardware 
inspections and functional testing activities need to be kept at a minimum by the Buyer, 
in all but the mostcritical or complex cases. Special case instructions and support 
need to be created regarding the witnessing of environmental and qualification testing 
by Buyer personnel. 

Source Appraisal- How well a source is performing is accomplished through analysis 
and review of reports, results of Source Control Plan audits, supplier Quality Conform- 
ance and percentage of product that is received "assembly-ready and f it-for-use 1 ' . The 
single most important action from Source Appraisal is the report to Management and 
Procurement of the results of the appraisal, on a timely basis. These same results must 
be fedback to the Supplier Management, both good and poor performance, and in the case 
of poor performance a request for corrective action and improvement. 

Corrective Action Programs- Whenever sub-standard quality items are identified the 
corrective action system is activated. Procurement Quality Assurance is responsible 
for detection and reporting discrepancies. Procurement is responsible for correction 
and initiation of action. Procurement Quality Assurance makes recommendations, Pro- 
curement makes decisions. Procurement Quality Assurance and Procurement, jointly, seek 
identification and elimination of the causes of problems at the Supplier's facility 
(Procurement Quality Assurance acts as the consulting expert, drawing on the expertise 
of Engineering and Other Functions, as needed, to further the investigation and resol- 
ution). Corrective Action is requested by Procurement, on every discrepant item, and 
it is coordinated at the Supplier by Procurement Quality Assurance. The reaction and 
importance attached to a corrective action request, by a supplier, is an indication of 
his integrity and sincerity. Followup on implementation and effectiveness of solutions 
is carried out by Procurement Quality Assurance, who reports results and confirms that 
the problem has been resolved, and no apparent side-effects have appeared, to Procure- 
ment, who then closes out the Corrective Action Request. 

Failure Investigations- Failures of all types are usually investigated by Engin- 
eering personnel, with Procurement and Procurement Quality Assurance assuming supportive 
roles. An investigation should be initiated on each failure identified as "Supplier 
Responsible". Whenever possible, failed item exhibits should be returned to the Suppl- 
ier so his personnel can analyze, first hand, and take action to eliminate the cause of 
the problem. If it is impossible to return the item (due to it’s destruction, etc.) a 
very detailed description of the circumstances existing at the time of failure, should 
be given. Procurement Quality Assurance representatives can coordinate the investig- 
ations and supply additional information and details which may be required by the 
supplier. The Procurement Quality Assurance representative can provide the Buyer 
investigators, an insight to the suppliers internal operations that, many times, will 
provide a clue as to which direction an investigation should take. 

Source Inspections and Tests- In process and final inspection of hardware and 
functional testing, when required, provide the basis for preliminary acceptance of the 
hardware, monitors configuration, records functional conformance data, and assures 
proper recording of deviations and Material Review Acceptances. This is partially 
accomplished through review and monitoring of quality records, inspection documents 
and test reports. The results of general surveillance activities and audits of in- 
process hardware contribute to the confirmation process and finally the witnessing of 
mandatory checks and tests involving the more critical and complex characteristics, 
completes the cycle of assurance. Results of all acceptance activities are reported 
to both the Buyer and the Supplier. Of course, the Corrective Action System is brought 
into play should there be any anomolies or defects. 

Receiving Inspection Plans- Proper procedures should oe carefully established for 
all major equipments. Material and equipment which has been Source Inspected and tested 
need not be given more than a cursory inspection to identify that it is the proper 
equipment called for by the purchase order and that there has been no shipping damage 
incurred. All other materials and equipment should have planned inspections and tests 
carried out to the detail necessary, as dictated by the criticality and essential nature 
of the item. 

Supplier Performance Ratings- The rating system must draw data from all areas of 
activity and then process this into a meaningful statement of the supplier's status. 



An often overlooked source of data is the post assembly and post delivery periods. 
Because, these, more than any other, indicate the degree of success attained from the 
previous planning and control efforts and provide indicators for future changes and 
improvements for the system. 

Probably one of the hardest lessions to learn regarding Procurement Quality Plan- 
ning and Control, is the "Plan" is never completed and "Control" is never assured. 
Because of the dynamic nature of change and technological advancement the Plan must be 
under constant review for several reasons: 

First, that product changes, both engineering and process, must be 
reviewed for effect upon present procedures and techniques. 

Second, an evaluation must be made periodically to support continuation 
of inspection and test performance. If, after a suitable period of time or 
quantity of units, no discrepancies are found, then serious consideration should 
be given to the use of some form of sampling or reduced sampling, or to the out- 
right elimination of the function. 

Third, on matured products or processes care must be exercised that 
careless or inadvertant mistakes and errors do not creep into the process and 
subvert the entire control network. The price of Quality is constant vigilance. 

Fourth, from time to time, products and processes should be reviewed 
for new technology applications and improvements . 

In summary, Procurement Quality Assurance Planning establishes the details of 
product complexity, levels of Procurement Quality Assurance involvement, needs for 
controls, and determines the manufacturing skills required. Procurement Quality Assur- 
ance participates in Supplier Capability appraisal, evaluation and selection. A coord- 
inated plan provides a vital communication link; a means of servicing a Supplier's needs 
for interpretation of requirements and timely coordination of problem resolutions; and 
a means of measuring performance. A close working relationship with the Procurement 
Quality Assurance representative provides a rapid means of establishing Acceptable 
Quality levels and making improvements through constructive criticism. The On-Site 
representatives provide a surveillance and monitoring function of Material Review 
activities and assistance with deviation and waiver requests. In addition, these 
representatives act as an early warning system, giving advance warning of schedule 
slippages, raising questions regarding possible manufacturing difficulties due to 
design or materials and alerting Buyer management, when there is laxity of Supplier 
management concern regarding problems. A well conceived Procurement Quality Assurance 
Planning and Control System can provide a continuing reporting and management support 
function which will highlight potential problem areas, contribute to the resolution of 
these problems and help assure the hardware delivered meets the requirements, is on 
time and is delivered at the most econimical cost. 
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ERROR REDUCTION IN THE JOB SHOP 


Reed Schafer, President 
Business & Industrial Consultants 
Box 226, Mooers, N.Y. 12958 


INTRODUCTION 

Small lot sizes often found in the job shop present a challenge 
in controlling quality since well known analysis techniques lose 
their statistical significance in measuring variation. Most techniques 
have been developed for large quantity manufacture. 

This paper will present a simple, inexpensive, possibly unconven- 
tional system which is helping to reduce scrap and salvage dollars and 
lost hours. It is of course important to remember that no system is a 
substitute for trained operators, capable processes and producible 
designs . 


PROBLEM 

The job shop under discussion exhibited a final inspection rejec- 
tion frequency of 30-40% with 10-15% of machine shop hours lost due to 
scrap and salvage. Management concern and requests for solutions were 
abundant . 


CAUSES 

Trend analysis of rejection causes indicated operator error was 
the classical maldistribution phenomenon of the few causing a dispro- 
portionately large effect on the whole. Further study indicated that 
lack of attention and lack of knowledge were the primary causes of 
operator error. Contrary to the belief of some, it was not "bad atti- 
tude" or "rookies" that were important causes. In interview after 
interview with operators where sizeable errors were discussed, it was 
evident that the twenty- five year veteran had more errors than he 
thought he should and that he had a very positive attitude in wanting 
to do better. This could have been expected since these same causes 
are well documented in the literature. Halpin mentions these same two 
causes of operator error in his well written classic on Zero Defects.! 

SYSTEM 

In formulating the system specification requirements some unique 
requirements began to emerge. A process control technique was needed 
particularly oriented to operator usage with a minimum of training and 
supervision. The technique had to be insensitive to part configuration, 
machine tool and years of experience. 

It was clear that the operator needed help in planning each job 
systematically in a manner that would discover errors of input to the 
operator as well as errors of output or operator caused. 

The thinking then turned to such questions as: 

How does the skilled operator keep himself out of trouble? 

What questions does he ask himself? 

What items does he check? 

When does he ask for help? 

How does he double check the job and himself to eliminate 

errors? 

At this point a list was developed which built upon the above 
questions with checkable items. The further the list developed, the 
more it resembled a pilots check list prior to takeoff. The thinking 
behind each was the same. 2 
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The list further developed along lines of job areas where possible 
variation could be expected which would cause error if not detected and 
resolved. The areas generally broke down into the following: 

1) Blueprint, specifications, process and bill of material. 

2) Tools, fixtures, gages, layout, setup and methods. 

3) Piece parts, materials, supplies. 

4) Process equipment, machine tool and special equipment. 

5) 1st piece inspection. 

6) Operator in-process inspection. 

So called Job Check cards (FIGURE 1) were then color coded and 
printed for each of the following processes: 


Machining 

Red 

Welding 

Black 

Assembly 

Green 

Electrical 

Blue 

Paint 

Purple 

Packaging 

Brown 

Crating 

Orange 


Color was used for easier identification as well operator esprit 
de corps. The cards are printed on oil resistant heavy paper with 
smear proof ink and a hole is punched for hanging. 

The Job Check "man" replacing the "K" in the word check was de- 
signed and became the logo for the entire program and appears wherever 
the system name of Job Check is used. (SEE LOGO PICTURED IN' FIGURE, 1) 

CORRECTIVE ACTION & FEEDBACK 

The problems discovered by the operator using Job Check analysis 
required a simple, effective and timely corrective action and feedback 
system. The four part Job Check Problem Report was developed with 
the following features: (SEE FIGURE 2) 

1) Job information such as part number and operator. 

2) Description of problem. 

3) Area responsible for corrective action. 

4) Description of corrective action by responsible area. 

5) Effective date of corrective action. 

6) Employee responsible for followup of corrective action 
effectiveness . 

The problem report was designed with one hard back copy and three 
thin"snap-out” copies. 


JOB CHEck'PROBLEM REPORT 



FIGURE 2 
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SYSTEM IMPLEMENTATION 

Prior to introducing the system to the operators, a foremen's 
supper meeting was called and the system was explained and questions 
and comments covered. Slogan banners using the logo enlarged were 
designed carrying slogans such as: "Job Check helps do it right" and 
"Job Check for 1st go OK". The banners were hung immediately before 
the meetings with the operators. All direct hourly people on all 
shifts attended with their supervisors. All Manufacturing Engineers 
dealing with floor problems were given their own meeting. 

The operator meetings, held in a conference room, were all alike * 
in format, lasted 20-30 minutes and covered the following points: 

1) System need, explanation and benefit. 

2) Function of the forms, routing and feedback. 

3) Operator's roll. 

4) Directions for filling out Problem Report. 

5) Question and answer session 

6) Appeal to participate for the benefit of all. 

A 99% favorable response was received from the operators. A 
pledge was made to the operators that management appreciated the 
problem input and it would see that positive and timely response was 
made. It was pointed out the operator had to make the effort to start 
the ball rolling. 


THE FUTURE 

Plans are in the development stage to provide motivational cita- 
tions and commendation awards to the operator for program participation. 
Shoulder patches showing the company name with a large blowup of the Job 
Check man in the center with the words "Job Check Award" emblazoned 
around the periphery were designed for an approximate cost of 75 cents 
each. Automatic pencils monogrammed with the company and program logo 


could be procured for about $1.50. Other awards included credit certi- 
ficates for purchase of shop tools through the company on discount. 

The plan was based upon the total number of legitimate problem 
reports turned in over a quarter. The first five awarded the operator 
a patch, 10 a pencil, 20 a certificate for $20 towards purchase of 
tools and each report over 20 added one dollar to the certificate value. 

Commendation certificates and awards would then be awarded to all 
eligible operators at a quarterly supper in their honor by the plant 
and/or division manager. This event would be covered by the local 
press and in-house newsletter. 


SUMMARY 

The Job Check system provides a job shop process control technique 
to help plan jobs and reduce errors by checking job input and operator 
interface. Minimum job performance is specified more clearly and vari- 
ation between operators is reduced. It provides a training aid for new 
operators . 

The system also provides a problem report which can be routed for 
corrective action and feedback to the originating department. It then 
becomes a permanent record file of problems for both the "detecting" 
as well as the "causing" departments. The followup assignment and 
effective date make it simple to seek improved effectiveness and time- 
liness of corrective action as well as the overall performance of 
service departments. 

1. Zero Defects - A New Dimension in Quality Assurance , by J. 

F. Halpin, McGraw-Hill, 1966. 

2. "Integrating Quality with Inventory Distribution" by J . F. 
LaDue, 1977 ASQC Technical Conference Transaction-Philadelphia, 
ASQC , Milwaukee 1977. 


LCS 720:20:439 



PARTS CONTROL 

A MANAGEMENT TOOL FOR QUALITY 


Donald L. Kear, Consultant 
Donald L. Kear Associates 
1706 Darst Avenue 
Dayton, Ohio 45A03 


INTRODUCTION 

Military standard MIL-STD-965, Parts Control Program , which was issued 15 April 
1977, introduces two parts control procedures — Procedure I and Procedure II. They 
are intended to reduce the proliferation of parts into the military inventory, thus 
reducing the costs of logistic support. An equally significant benefit, however, is 
their contribution to the quality of the system or equipment being designed. Engineer- 
ing design managers and quality managers should consider the advantages of using the 
parts control program as a management tool for quality. 

The procedures offer two different management techniques with each one capable of 
being tailored to meet the needs of a particular contract. Procedure I as a general 
rule will be applied to most contracts, whereas Procedure II with its parts control 
board (PCB) concept normally will be used on systems contracts only. 

Although the program controls standard as well as nonstandard parts, the part 
approval procedures have been simplified over past procedures by the introduction of a 
Program Parts Selection List (PPSL). The PPSL more clearly identifies the parts that 
are approved for design selection than did previous procedures, and should overcome 
the problems associated with identifying standard parts. 

Further, the program is compatible with the current emphasis on using commercial 
parts, and offers many other cost avoidance features. 

MIL-STD-965 EVOLUTION 

The requirement for the MIL-STD-965 program evolved over a period of more than 
20 years. In 1957, the results of a study on reliability were reported by the Advi- 
sory Group on the Reliability of Electronic Equipment, in which parts were identified 
as a major factor in field failures. Complying with the recommendations of the 
report, a task group conducted a more detailed study, and issued a Parts Specification 
Management Report in 1960. This report recommended up-dating parts specifications to 
establish measureable reliability requirements, thereby making the specification 
useable for design. 

This recommendation was saying, in effect, "STANDARDIZE DURING DESIGN" to achieve 
quality and reliability. Subsequent studies by the Logistics Management Institute, 
the U.S. Air Force Panel 29, a DoD Parts System Task Group, and finally the House Sub- 
committee on Government Operations in two separate studies reached the same con- 
clusion. To implement the recommendation, the Office of the Assistant Secretary of 
Defense for Installations and Logistics ((OASD (I&L)) directed the military depart- 
ments to adopt the parts control program recommended by the DoD Task Group by 1 July 
1971 for all electronic procurements. 

In the meantime, the Air Force had initiated the parts control board (PCB) con- 
cept around 1967, as prescribed in MIL-STD-891 (USAF) , Contractor Parts Control and 
Standardization Program. Under this concept, the prime contractor for a weapon system 
was delegated the responsibility of standardizing parts during design. This required 
the contractor to assure the Air Force that only those parts of acceptable quality 
were used. Thus, while system standardization was the primary objective of the parts 
control program, quality and reliability were also major considerations. 

During much of the same 20-year period, a second parts control document was ex- 
tensively used — MIL-STD-749, Preparation and Submission of Data for Approval of 
Nonstandard Parts . While the nonstandard part approval procedures of the two docu- 
ments, (MIL-STD-891 and MIL-STD-749) basically conformed to the procedures of MIL- 
STD-749, other standards, specifications, and contract exhibits introduced variations. 
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As of January 1978, 35 different documents have been identified. The effect of the 
varying procedures was to cause variations in the quality of parts used in design. 

Along with the different procedures, the nonstandard part approvals varied 
greatly. A contractor might receive approval for a part from one procuring activity, 
but have the same part disapproved by another procuring activity. This is understand- 
able when the different kinds of applications are considered. A part which is accept- 
able for an environmentally controlled ground site may not be acceptable in an air- 
craft which subjects the part to different environments and stresses. From a quality 
and standardization aspect, however, parts used in similar applications should have 
been approved regardless of the procuring office making the evaluation. 

When the OASD (I&L) order was given in 1971 to apply parts control to all elec- 
tronic systems contracts, the Defense Electronics Supply Center (DESC) was assigned 
the task of reviewing nonstandard parts for the Army and Navy. (DESC had been con- 
ducting such a review for the Air Force for several years under an agreement between 
Air Force and the Defense Logistics Agency.) In this review, it was determined that 
more than 50 percent of the nonstandard parts proposed by contractors were more than 
adequately covered by military specifications. In most cases, the parts were not 
equal to the quality of the specification parts, and in only a few instances did con- 
tractors attempt to justify the parts on the basis of quality. The usual justifica- 
tion was "no standard part available," meaning, of course, that no standard part 
existed exactly like the nonstandard parts. 

Moreover, some designers mistakenly believed that any part covered by a military 
specification was "standard." Or, that any part which was "standard" to industry was 
also "standard" for a military contract. Thus, many so-called standard parts were 
used without approval of the procuring activity, even though they did not measure up 
to expected quality requirements. This situation was discovered during the review of 
parts used under MIL-STD-891 (USAF) . 

System standardization was accomplished under MIL- STD- 891 (USAF) by listing parts 
in a PPSL, which was divided into a Standard Parts Section and a Limited Application 
Section. Of the so-called "standard parts" proposed for listing, approximately 23 
percent were obsolete, or their quality characteristics were inadequate to meet con- 
tractual requirements.^) In these cases, the parts had been selected from military 
specifications, and had been chosen on the basis of lower costs. Sometimes the con- 
tractors were exhausting stocks used on previous contracts. Such decisions may be 
economically justifiable to contractors, but the lower quality equipment increases 
the life cycle costs, and these costs more than offset the savings. 

Another contributing factor to properly identifying standard parts was the vari- 
ety of meanings for the term "standard part” as used within the Department of Defense. 
Although recent changes have clarified the term, it was possible to- have parts which 
were "standard" for design, parts which were "standard" for production, and parts 
which were "standard" for supply and maintenance, all with different criteria and 
implications. (2) Recognizing the difficulty in a universal definition, MIL-STD-749 
left the criteria to the governing general military equipment specification, or 
contract. 

Even MIL-STD-891 (USAF) complicated the definition. In this use of the term, any 
part which was acceptable for design application throughout the entire system was 
"standard" for the system. Once the part was listed as "standard" it could be used 
without further justification, even though it might be a commercial part. Parts con- 
trol in this sense was intended to standardize parts within a system. 

After more than 10 years of acquiring data on the quality of parts used in de- 
sign, and after seeing the problems created by the existence of so many parts approval 
documents, the decision was made to establish a single parts control program, and to 
issue MIL-STD-965 with controls for both standard and nonstandard parts. Further, 
experience with logistics and quality problems on other parts such as fasteners man- 
dated expanding parts control to include all parts — not just electrical and elec- 
tronic parts. 

In a point not related to quality, but one which is important to understanding 
the subject, clarification should be made about the term "all parts." MIL-STD-965 
excludes structural members and machined parts that are specifically fabricated for 
a particular application, and are not adaptable to other applications. Thus, for 
example, a bracket made to mount a card assembly is exempt from the parts control 
program. 



The problems associated with the definition for standard parts has been resolved 
in MIL-STD-965 by renaming the two sections of the PPSL. Section I is titled "General 
Application Parts" which allows any part to be listed regardless of whether its origin 
is from the commercial field, or from a military specification. This title is con- 
sistent with the intent of the section, and likewise is consistent with the title of 
the second section, "Limited Application Parts." 

Another significant change has taken place in MIL-STD-965 from previous parts 
approval procedures. Contractors are encouraged to call Military Parts Control Advi- 
sory Group (MPCAG) engineers at the Defense Industrial Supply Center (DISC) and DESC 
to discuss parts requirements. This procedure has been effective for some time in 
reducing the number of nonstandard part submittals, and has improved the quality of 
parts being used in design. 

More significantly, engineers now communicate with one another about the charac- 
teristics of parts, and about problems that might be encountered with some parts. The 
designer now has more information in enabling a decision to (1) design around the de- 
ficiencies, (2) specify quality requirements in part procurement specifications that 
will assure receiving acceptable parts, or (3) select other parts. 

QUALITY/PARTS CONTROL CASE HISTORIES 

Historically it has been argued that any standardization activity such as parts 
control detracts from quality and reliability. The following case histories are 
offered as evidence that parts control can contribute to quality, and help avoid 
costly system or equipment down-times, production costs, and field maintenance costs. 

In a research of reliability and its impact on life cycle cost (LCC) , conclusions 
were reached that the LCC objectives can best be met by standardizing parts, provided 
the appropriate parts are used. Two of the radars under study were similar in acqui- 
sition cost with the APQ-120 at $24. 2M, and the APQ-113 conting $17. 1M. The spares 
acquisition costs were higher for the APQ-113, but the monthly logistics and main- 
tenance costs were $299 for the APQ-113 compared to $746 for the APQ-120. These 
monthly costs represent a differential of 2.5:1. (3) 


The report shows the APQ-120 demonstrating a 4.3-hours MTBF, or 50 percent of its 
requirement, while the APQ-113 exceeded its requirement by 12 percent with a demon- 
strated 152-hours MTBF.™' The number of parts used in the APQ-113 were limited to 
approximately 10,700 parts, whereas the APQ-120 used 13,500 parts. (5) 


The report concluded that standardization using the wrong parts adversely affect- 
ed reliability. The APQ-120 standardized on military specification parts, but the 
parts were being superseded by established reliability (ER) parts. (The contracts 
were in effect during the period when the conventional parts specifications and the ER 
specifications were both listed in the military bookform standards. Both kinds of 
parts were considered standard.) The report recommended that all parts should be 
screened to a reliability requirement such as testing extra (TX) , ER, or to the new 
general specification for microcircuits, MIL-M-38510. ^ 

A second example shows that unwarranted costs are incurred during production if 
substandard quality parts are used. In an analysis of failures of microcircuits used 
in two F-15 aircraft equipment, McDonnell Douglas Aircraft Company reported the remov- 
al rate of microcurcuits in card assemblies was 2.7 times greater for nonstandard than 
standard MIL-M-38510 microcircuits.^) The major reasons for the difference in quali- 
ty were attributed to the less stringent vendor catalog test conditions, and the 
closer vendor surveillance by government inspectors for the MIL-M-38510 microcircuit 
procurements . 

While no production repair costs were cited in the report, they could be signifi- 
cant when you consider that 13,465 nonstandard microcircuits were processed through 
manufacturing during the 12-month study period. 

This final example compares before and after standardization of a navigation 
system. In the system produced without a formal parts control program, more than 250 
part types were used in design. These types represented a population of 4000 parts. 
The reliability record of the system was excellent demonstrating 1139-hours MTBF 
against a predicted 1190-hours MTBF. 

Under an Air Force contract a new version of the system was developed. The con- 
tract imposed a formal parts control program, and set a goal of no more than 3700 part 



population. The completed system improved on the goal by 16.8 percent with a part 
population of only 3077 parts. Although no goal was set for the number of part types, 
the system used only 183, thereby improving by 26.8 percent over the nonstandardized 
system. 

The contractor was apprehensive at the outset of the contract about the effect of 
parts control on reliability. Nevertheless, management control was exercised and 87 
percent standardization was achieved. The 13 percent nonstandard parts were mostly 
microcircuits not covered by MIL-M-38510. The parts control program paid off when the 
reliability demonstration exceeded expectations. So few relevant failures were re- 
corded during the first 644 hours of test that the demonstrated MTBF should easily ex- 
ceed the predicted 1474-hours MTBF. 

COST AVOIDANCES 

Although the scope of the parts control program covers a broad range of parts, 
MIL-STD-965 provides many cost avoidance features that should make the program very 
attractive to a quality manager. Foremost is the provision that permits tailoring 
of the procedures to meet the objectives of the procuring office. Thus, for example, 
parts that do not impact quality and reliability can be exempted from control on a 
specific contract. 

Responsiveness to contract schedules is another cost avoidance aspect not pre- 
sent in previous parts approval programs. Telephone contacts between parts engineers 
has already been cited as one method of expediting replies. When a written submittal 
of a nonstandard part request is required, the procuring office is obligated to reply 
within a time schedule established by the contract. If the reply is not received the 
part is automatically approved. Many procuring offices have adopted the practice of 
replying only when disapproving parts. 

A third cost avoidance feature is associated with the telephonic procedure. 

Under Procedure I, parts requests normally will be handled by telephone with the MPCAG 
at DESC, or at the DISC. The MPCAG will confirm its recommendation in writing to the 
contractor, and to the procuring office. This procedure eliminates a cost to the con- 
tractor for each submission. Even the need to confirm a nonstandard part request is 
eliminated when agreement is reached to use a standard part. 

Additional reduction in documentation costs accrue from a significant reduction 
in drawing requirements. A drawing will be prepared only when a part is selected for 
design from an approved PPSL. This precludes preparing a drawing that has less than 
50 percent chance of being approved. (More than 50 percent of the nonstandard part 
requests are disapproved.) With drawing costs ranging from $500 to $9000 each, and 
with 11,676 nonstandard part requests being disapproved at DESC alone in FY1977, this 
conservatively avoids more than $6.5M for electrical and electronic parts drawings.^) 

Field maintenance of failed equipment represents a cost that can be reduced 
through parts control. According to one study, the cost of field maintenance ranges 
from $225 to $408 for each action, at an annual maintenance cost amounting to $6.1 
billion. (10) With maintenance costs averaging more than $300, improved quality 
through parts control could significantly reduce the maintenance budget. 

COMMERCIAL/MILITARY PART QUALITY 

You may have observed that the parts control program seems to be on a collision 
course with the drive for increased use of commercial parts. However, as has already 
been stated, MIL-STD-965 allows for the use of commercial parts. No conflict should 
exist between the objectives of the parts control program and the objectives of the 
Government agencies advocating the use of commercial parts. How well the conflict is 
avoided is dependent upon how effectively quality control becomes involved in the part 
selection decision process. From my vantage point as a quality control consultant and 
as a standardization consultant, quality control must participate in the decision 
making to assure consideration of the long-term effects of the decision. 

Expanded use of commercial parts is advocated in the Government on the basis that 
military specifications tend to apply the same criterion to all applications. "Gold- 
plating," "overkill" and "cost-drivers" are some of the terms given to the specifica- 
tion requirements, and in some cases appropriately so. However, careful consideration 
must be given to the impact of reducing quality requirements. For example, a recent 
paper cited high reliability electronic parts as being unnecessary for an attended 
ground radar system. The author stated the difference between commercial and military 
microcircuits as being in the screening and burn-in done to eliminate the imperfect 


devices that slip through the manufacturing process. It was suggested that the system 
could do its own screening, and the better quality military microcircuits could be 
used in repair. It was further suggested that the increased failure rate might even 
provide a better training experience for a maintenance crew. (H) 


Screening and burn-in do contribute to the better quality of the military micro- 
circuits, but these are not the major factors. The military microcircuits are sub- 
jected to rigorous process control, tighter electrical parameters are specified, and 
the devices are inspected under tighter test conditions. 

Perhaps the suggestion about increased failure rates giving maintenance crews 
better training was made with tongue-in-cheek. However, such a concept would be 
extremely costly, and would result in increased system down-time. Consider an airport 
radar with a 10,000 part population, and having a two-percent failure rate. That 
radar would be out of service on at least 200 occasions, and at $300 for each repair 
the maintenance costs would approximate $60,000. This is an exorbitant training 
cost, but more significantly the system could be out of operation 66 percent of the 
first year of operation. 


Commercial parts in such an application do not seem to be the logical choice. 
However, waiver of some quality requirements of the military microcircuit specifi- 
cations for this application might be appropriate, and could reduce costs. The high 
altitude requirement might be one such requirement. These are the kinds of decisions 
that, in my judgment, must be made to assure that commercial parts are used when 
appropriate. Quality should not be sacrificed on the altar of cost savings. 


CONCLUSION 

The premise that parts control contributes to equipment or system design is more 
than substantiated by data accumulated in the 10 years prior to issuing MIL-STD-965. 
Standardization does not inhibit technological advance when applied properly, and the 
tailoring permitted by MIL-STD-965 gives designers wide latitude for selecting quality 
parts . 


Significant cost avoidances are achievable under a properly managed parts control 
program, with a major portion of the benefits being in reduced field maintenance cost. 
Such a potential merits the attention of the quality control manager, and removes 
parts control from the sole realm of the standardization manager. 
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NO FAULT SCRAP AND REWORK ACCOUNTING 


D.N. Ekvall, Director, Quality Assurance 
Leeds £ Northrup Company 
North Wales , Pennsylvania 


To the Accountant, reporting scrap and rework is obvious and straight-forward 
but to those who must make those charges the fear of blame and the uncertainty of 
repercussions can send a shudder aown the spine. As a result when conventional 
approaches are used, scrap and rework charges go underground. My investigation of 
this subject in several companies indicated that scrap and rework is at least as 
much an emotional problem as it is an accounting one. No matter who is involved, 
foreman, supervisor or manager, the question is always asked -"where shall I charge 
this scrap?" When what is really meant is - "how can I make this problem go away 
without hurting me." I have uncovered a number of ploys which are in current vogue. 

1. Charge the scrap to a general ledger account - they are always hard to 
track and one more small transaction is sure to get lost. 

2 . Ignore the charge and eventually the accountants will have to write it 
off to balance the books. 

3. Charge a non-existing account - by the time the "bookies" unravel it they 
won't know what to do - and if we're caught, we can always plead ignorance - 
they would believe that. 

4. Charge it to the job - while the cost ratio on this job may be poor, in the 
long run it will average out. 

5. Charge it to another job in process - that is almost impossible to detect. 

There are other more ingenious schemes too - but they all have one thing is 
common and that is - no understanding of the objectives that need to be served by a 
reliable record of scrap and rework. The answer is not to detect or foil the 
schemes because for every one blocked, the men in the trenches will develop a new one 
just a little more difficult to detect. This will lead to a series of games to see 
just who can outsmart who. The loser will be the Corporation without adequate data 
to recognize and correct its management problems. It will result in poor quality 
products and keep the company from measuring its progress along the path to 
excellence . 

The objective of reporting, collecting and analyzing scrap and rework costs is 
to be able to recognize specific high and recurring costs. By "Paretoizing" this 
data we are able to measure the magnitude and trends of the situation and to use our 
limited resources for corrective action where it will do the most good. It is truly 
amazing 1 to me how much interest there is in scrap and rework costs and how many 
points of view can be exposed when you attempt to change the reporting situation. 

The accounts, the manufacturers, and the quality controllers, each have their own 
opinion, they each recognize the importance of the control of scrap and rework, but 
wili be quite cautious about upsetting the existing norm, whatever that is. 

The accountants interest is primarily financial, they will favor action which 
will reduce this cost but will have to be persuaded that it is worthwhile before they 
will be willing to change their accounting system. 

The manufacturing interest is defensive, while they too would like to reduce the 
cost of scrap and rework, they have in the past, been blamed for all of it regardless 
of the cause, i.e., design, material, storage protection, practices, as well for 
those causes over which they do excercise control. 
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While all accounting systems must collect the cost of scrap and rework to 
balance the books, unless the quality interest has been considered in that system 
design, the data may not be in a format which can be used to get clear visibility 
on this important problem. The problem quality managers face is developing and 
selling a system which will accommodate all interests without compromise, and that 
is where the "No-Fault" concept comes into play. 

The "No- Fault" concept is as follows: 

1. Charge all scrap and rework to accounts for which Q.A. is responsible. 

2. The data should be coded to indicate where scrap and rework are identified 
(not who is responsible). 

3. Analyze the data to give the important specific parts visibility so 
that effective corrective action cycles can be implemented. 

4. Give credit to those who find the cause and solve the important scrap 
and rework problems. 

The whole concept is a positive way of dealing with a reality. Scrap and 
rework data have already cost the corporation once . The positive attitude toward 
it opens up an opportunity to improve. It allows management to manage and control 
the number of times we live through the same mistake. Quality managers and their 
superiors may wince at having all of ,-^he corporation's scrap and rework charged to 
quality, but recognize these important facts :- 

1. The costs of scrap and rework have already been incurred by the corporation 
and it must be charged somewhere, i.e., the cost to the corporation has not 
changed, it's just been redistributed. 

2. Scrap and rework is probably the richest vein of the quality control 
"gold mine" and in charging it to "quality" you have given it to a party 
with a vested interest in reducing that cost. 

3. Charging all the corporation's scrap and rework in one place with adequate 
coding, gives visibility to the real magnitude of the situation and quality 
control the ability to generate many different actions to identify what 
would otherwise be hidden knowledge of its true operation. 

In my first application of "No-Fault" scrap and rework accounting, I dealt 
with scrap and rework generated in inventory orders. It was easy to identify three 
basic sources of that cost. 

1. Scrap and rework due to material rejected by inspection. 

2. Scrap and rework caused by defective material supplied on an inventory 
order. 

3. Scrap and rework generated when material is found defective in stores. 

These three broad but simple categories gave a very quick first cut to 
classify our data and identify the basic causes of why scrap and rework occurred. 

The same kind of basic classifications can be developed for other forms of 
manufacturing orders, i.e., customer orders, tool orders, purchased material orders. 

It should be recognized that in developing an accounting system for scrap and 
rework, care should be taken in developing interface between quality, manufacturing, 
accounting, and the material management system. Remember that you do have an 
interest in those transactions which return material to stores, with those 
transactions which dispose of materials, for those transactions which charge time 
for correcting work, in effect for all of the charges that are beyond the standard 
cost envisoned when the job was engineered. 

Provision must be made for issuing additional material to perform rework for 
repair and for returning good material to their inventories as appropriate . I 
will not spend time in this paper discussing the developments of forms and formats 


for each accounting and manufacturing system has already been tailored to the 
products and processes germain to its products. My point here is that coordination 
is necessary before the fact to set up the formats, to set up the forms, to agree 
on the transactional tracks and to develop supporting procedures instead of 
arguing over who is at fault for the high cost of scrap ana rework. 

The whole thesis and concept of the "No- Fault" scrap ana rework program 
described in this paper is to recognize that no good can come to the corporation 
by placing blame - and only by providing the non-emotional "No-Fault" type of 
system, can scrap and rework data flow freely into management documents that allow 
for effective corrective action - the best utilization for that high price already 
paid. 
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ULTRA SMALL DIAMETER BORESCOPES 


DESMOND H. MURPHY, SALES MANAGER 
OLYMPUS CORPORATION OF AMERICA 
INDUSTRIAL FIBEROPTICS DEPARTMENT 
NEW HYDE PARK, NEW YORK 


INTRODUCTION 

Rigid borescopes using conventional lens systems are generally ade- 
quate from the 10mm to 4mm outside diameter sizes, but image quality and 
brightness decrease significantly when small diameter access ports re- 
quire very thin borescopes. 

New developments in fiberoptic technology now make possible high 
quality rigid borescopes to meet such requirements. 

THE SELFOSCOPE 

Regarded as the smallest fiberoptic rigid borescope currently 
available, the Selfoscope is equipped with a stainless steel optical 
insertion tube of only 1.7mm (.067") in diameter. Working length models 
are either 110mm (4") or 176mm (6.9"). 

The Selfoscope has a deep depth of field from 1mm to infinity and is 
ideally suited for close-up observation in a small cavity. The magnifi- 
cation factor at 1mm (.04") is 30x; at 25 mm (1"), 1:1. 



FIGURE 1. 
THE SELFOSCOPE 
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Image Transmission 


Clear resolution is obtained by transmitting the image through a 
single SELFOC fiber which is less than 1mm in diameter. The SELFOC ® 
fiber, jointed developed and manufactured by Nippon Sheet Glass Company, 
Ltd. and Nippon Electric Company, Ltd. of Japan, is a uniquely formulated 
glass fiber with diffused metallic ions. The result is a graded index 
fiber; its index of refraction decreases radially from the center to the 
periphery according to a parabolic equation. This newly developed wide- 
angle, single rod lens is coated with a layer of light carrying fibers 
for illumination. 
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FIGURE 2. 

IMAGE TRANSMISSION 


Illumination 


The light guide is constructed with continuous fibers from the 
distal tip of the scope to the external 150W, high intensity, cold light 
source. This continuous construction of the light guide is a critical 
illumination factor in a borescope of such small diameter. If the light 
guide was a separate and detachable unit, illumination would be reduced 
approximately 50% due to losses at the connection interface. Since the 
light guide fibers surround the objective lens, maximum illumination is 
focused on the viewing area. 



FIGURE 3. 

REMOTE COLD LIGHT SOURCE 
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w SELFOC is a registered trade mark. 



Direction of View 


The Selfoscope's direction of view is either direct, or side, or 
forward-oblique. Both the direct and side viewing models have a 55° 
angle of view; whereas, the forward-oblique model has an angle of view of 


75° . 


Direct 

View 


Forward 
Oblique View 



DISTAL END 



FIGURE 4. 

SELFO SCOPE VIEWING MODELS 


Photographic Capability 

For photographic capability, a SLR 35mm automatic exposure camera is 
attached to the Selfoscope via an adapter. No camera lens is required 
since the optics within the Selfoscope are utilized for maximum photo- 
graphic results. 



FIGURE 5. 

SLR 35MM AUTOMATIC EXPOSURE CAMERA (BODY ONLY) 
WITH ADAPTER ATTACHED. 


Similarly, the Selfoscope may be attached to a CCTV system for magnified 
viewing on a monitor. 



Applications 

The Selfoscope is gaining wide acceptance in the quality control 
departments of the electronic, drug, scientific instrumentation, small 
motor, valve and investment castings industries. For example, visual 
inspections are now accomplished inside turbine blades with the 
Selfoscope, checking for cracks or other defects of a critical nature. 
This type of inspection can not be easily accomplished with other NDT 
methods such as x-ray; the intricate, internal construction of the 
turbine blading may interfere with a true reading. 

Further, the Selfoscope solved a critical drive-belt mechanism 
problem for a quality control department in the vacuum cleaner industry. 
Not discovered until after final assembly, the problem was to find out 
which of the drive-belt mechanisms in several thousand vacuum cleaner 
units shipped to the field were faulty without expensive and time 
consuming call-back and tear-down. The simple removal of a 5/64 set 
screw in the housing allowed the insertion of the Selfoscope which 
zeroed-in to the problem area. As a result, the entire lot was inspected 
in the field in two days with only a very small percentage of the units 
subject to re-ca.ll. 



FIGURE 6. 

View inside a Turbine Blade. 
Photo shows the Selfoscope' s 
great depth of field; 
separators in the foreground 
the honeycomb in the back- 
ground . 



FIGURE 7. 

View of A Circuit Board 
Inside a Housing. 

Photo shows the Selfoscope 's 
magnifying capability. 


CONCLUSION 

Visual inspections inside very small cavities with the Selfoscope 
are fast, simple and inexpensive in comparison with other NDT methods. 

In this age of miniaturization, a finished product may be so compact 
that a quality control inspection of its internal components is difficult. 
The solution to this type of problem may very well be the ultra small 
diameter Selfoscope. 


LCS ; 


750 70 438 




QUALITY AUDITS IN SUPPORT OF SMALL BUSINESS 


Walter Willborn 
University of Manitoba 

INTRODUCTION 

The Quality Control Handbook (1) states: "Because standards for audits are 
still in a state of evolution the written material is not complete and there remain 
important unanswered questions." 

One of these questions concerns quality audits for small, independent 
enterprises. This paper deals with an innovative audit service especially designed 
for small business. 


BACKGROUND 

Inadequate planning and control of operations ranks as a major cause for small 
business failures. This critically weak management extends to quality control. Many 
owner/managers do not consider quality of design, conformance and performance an 
issue that urgently requires their attention. In a prevailing environment of 
informality and personal relationship one still assumes "good workmanship" sufficient 
for assuring quality, although often it remains undefined and there' fore 
uncontrollable. Technological advances along with other socio-economic changes have 
changed needs and conditions for quality control in small enterprises 
drastically. 

One cannot expect that small business will improve its quality control without 
aids from the community. In realization of the importance of a healthy and 
competitive small business section in a free enterprise economy various kinds of 
management assistance programs already exist. The Small Business Administration (2), 
as the best known supporting agency, for instance, has helped effectively. In the 
area of improving quality control, however, the available assistance is minimal, 
mainly because small business operators do not perceive the need to, and the benefit 
of modern methods. For outsiders, free enterprise principle do not allow undue 
interference with management. Still, in this case initiative had to come from 
elsewhere than the small business; at least in Manitoba. 

Quality audits appeared to us as a feasible and useful external aid to small 
business management. Before designing a suitable audit service we explored the 
following questions: 

- What is a workable definition of 'small business' and what is its scope? 

- What are unique features of quality control in small business? 

- What are the limitations of current quality audits in small business as a 
guide for management? 

SBA defines "small business" as "firms which are profit seeking, have 500 and 
fewer employees and are not dominant in their field." 97 percent of all U.S. com- 
panies fall into this category (3). 

In recent years growing importance of quality control in small enterprises arose 
from the following: 

- technology related to products, processes and equipment has become more 
complex and demanding to producer and consumer. 

- customers require compliance with generic standards for quality contro’ 
programs . 

- other small companies have already established effective quality control 
procedures and gained competitive advantage. 
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- legislation for consumer protection, higher incidence of product 
liability claims and reduced rates for insurance require evidence of an 
adequate quality control program. 

A manager /owner finds little help from his traditional management advisor, the 
accountant, with regard to quality control. Quality auditors that represent 
government and customers' interest cannot act freely and independently. These 
quality audits are for the small business operator of limited value by the following 
reasons : 

- main purpose of the audit is to establish evidence and to attest rather 
than to assist, 

- they are not requested by the owner/manager in awareness of need for and 
possible benefits from improved quality control. The audits are more or 
less imposed upon them. 

- they are conducted infrequently, in conjunction with a specific contract 
or legal act, and without due regard to small business conditions. 

THE PROBLEM AND STRATEGY 

On one hand, mounting pressures for improved quality control and insufficient 
service on the other caught small business in the middle. It became obvious that the 
main problem rests with the owner/managers in that they are reluctant to act, and to 
initiate assistance. In order to overcome this mainly psychological problem we 
proceeded very cautiously (4). In our strategy and audit service we remain cognizant 
of the following characteristics of typical owner/managers of small enterprises: 

- they tend to operate by "common sense", tradition, trial and error and 
day to day problems. They are not professional managers. 

- they are hard working, self-reliant and energetic individuals and do not 
accept advice from outsiders easily. 

- they maintain close personal relationships with employees, business part- 
ners and the surrounding local community. 

With this personality in mind we arrived at a strategy by which we approach 
owner /managers first on a community wide basis and later individually. A "community 
quality audit" precedes and prepares "small company audits". Figure 1 shows a design 
of a comprehensive quality audit service with two interlinked sub-types that is 
geared to take advantage of close relationships between community and small business. 
In this scheme owner /managers hold a central position as an active and decisive 
participant in the quality audit. 



Fig. I. Quality Audits at Community and Company Basis 


COMMUNITY QUALITY AUDIT 


Main purpose and objectives of this new kind of quality audit are as follows: 

- generate information on the status quo of quality control, scope for 
improvement, need for assistance and availability of resources, 

- create an awareness for quality control in the community and its small 
business section, 

- establish first contacts with small business operators and explore the 
clientel for company quality audits, 

- conduct research and provide a report for a steering committee. 

An initial base-line audit explores the status-quo on a limited scale and in 
general terms, provides learning experience for auditors and auditees and creates 
interest and further support in the community at large. A standard for subsequent 
community audits in form of an expected and desirable status of quality control in 
the community will gradually evolve with time. 

Community quality audits constitute "audits" and not just "surveys" because they 
aim at: 


- establishing a general standard for a community wide quality program and 
quality control improvement program, 

- appraising and improving the effectiveness on such programs, 

- implementing the policies of the steering committee, 

- ensuring that human, financial and material resources are adequately 
known, utilized, and developed, 

- initiating remedial action and follow-up audits at both community and 
company levels. 

With regard to the classification of quality audits by Mills (5) community 
quality audits describe a location and system oriented audit rather than a product 
and process audit. 

Who should initiate such community quality audits? This responsibility will 
fall on the steering committee. This committee should have a strong representation 
from small business. While government officials will not qualify as independent 
auditors they can contribute as committee members along with those of other 
organizations . 

The baseline audit sets the stage for the entire audit program. Actually, any- 
body sufficiently motivated, informed and able to attain the cooperation of others 
can pioneer such an audit in the local community. The author, being involved in the 
local quality movement for some years as educator and executive members in the ASQC 
Section of Manitoba originated the first community quality audit. The Manitoba 
government had conducted similar so-called Productivity Audits and joined in the 
project along with individuals from the university, ASQC and other local organiza- 
tions. A faculty member of the local university has particular advantages to act as 
"neutral" leader. — In the U.S. such audits qualify for grants under Bill S-972 for 
Small Business Development centers at universities. — 

In our Manitoba audit we applied a fairly conventional approach. We used 
questionnaires and question checklists. Of 700 small manufacturing establishments, 
about 40 percent responded. We checked a great number through subsequent interviews. 
These we carefully prepared as we wanted not only to gather data, but at the same 
time establish first personal contact with a hopeful future client for a company 
quality audit. With the exception of the initially mailed questionnaire the proce- 
dure compares with that of a normal company audit. 


We tabulated the answers and statistically evaluated them for proper analysis. 
Most interesting and revealing were also answefs to our open questions. It became 
quite clear that many of our owner/ managers recognize the importance of improved 
quality control. They want better training opportunities and information service in 
order to overcome quality control problems. With regard to company quality audits 
many were prepared to submit their current practices to an appraisal and as a 
prerequisite for improvement. These audits, however, were to be conducted by an 
independent, well qualified auditor, who is neither employed by another company nor 
the government. At the end of the community quality audit we knew our potential 
clientel for company quality audits quite well. 

A report on the community quality audit we forwarded to persons in local 
associations, government departments and educational institutions along with a 
request to join a steering committee. All of them responded en thus i as t ica 1 1 y , 
somewhat to our surprise. This showed that at least in our province quality control 
is an issue very much on the mind of many people. This independent "citizen 
committee" elected the author as its chairman and provided a well appreciated and 
unique forum for discussing concerns for better quality control in the business 
section of the local community. With focus on improvement of quality control the 
committee developed its function and role as a policy making and management body. 
(See figure 2.) 
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Fig. 2 . Community Quality Audit 


The report from the community quality audit provided the major basis for discus- 
sion and constructive recommendations. As a result of the partly quite demanding 
work of the committee a government position for a coordinator of quality assurance 
has been created, a university certificate course in quality assurance management ar- 
ranged with a surprisingly high enrolment and finally quality audits in small 
companies have commenced. 


SMALL COMPANY QUALITY AUDIT 

As mentioned before, one of the major objectives of our community quality audit 
was from the outset to contact small business operators, to inform them about quality 
control, to actively involve them in a broadly based quality audit and thus to prompt 
them in their request for an independent company quality audit. What we real- ly 
were looking for was a genuinely interested clientel for these audits which once 
satisfied with our service would induce others to follow and request the same 


service . 


The following reasons caused owner/managers to participate in quality audits: 


- through the preceding community wide project they simply wanted to 
continue in a project with which they identified themselves already; they 
continued in the effort, 

- they learned from meetings and the report enough about quality control 
with regard to their immediate local environment in a most realistic, 
simple and practical manner. Naturally, they wanted to know more details 
about the comparison of their current company practices with the 
prevailing status in the surrounding local community. 

- they acknowledged and appreciated the steering committee and its 
individual members as competent and reliable trustee for conducting and 
surveilling company audits and independent quality auditors. 

- they became acquainted with some of the quality auditors who actively 
participated in the community quality audit. 

- they realized that benefits will easily exceed the costs partly because 
public grants for the audit were made available. 

- they recognized that because of the prevailing close personal ties in 
small business to bring about more formal and restrictive quality control 
procedures would be difficult without the help from a competent quality 
auditor as an outsider. 

In more detail, owne r /manager s expect from this company quality audit the 
following service and outcome: 

- an interpretation of the report on the community quality audit with 
regard t:o the individual company, 

- an assessment of current quality control practices of the company and 
recommendations for simple, practical and short term improvements, 

- an assistance in setting-up such an improved quality control program, 

- a documentation of this quality control program in an easily understand- 
able language and form, that facilitates implementation and maintenance 
through self-audits by the owner/manager , 

- further convenient availability of the auditor whenever needed. 

The owner/manager does not want an audit that completely by-passes him/her and 
results in a report. Obviously, a quality audit geared to assist a small business 
operator has different features when compared with those in large companies. The 
auditor of a small and independent enterprise must work very closely through all 
phases of the audit with the owner/manager in order to be accepted and to render 
useful service. 

Consequently such an auditor must have acquired additional qualifications such 
as : 

- knowledge and experience in small business management along with a 
sincere appreciation of small business values, 

- ability to communicate well with small business operators, to listen, to 
understand, and to work with them rather than for them, 

- competence in adopting generic program standards, specific procedures and 
methods to needs and conditions in small business. 

Individual phases of the audit in which client and auditor have to work together 
are depicted in Figure 3. From the discussion of the community audit report and the 
owner /manager ' s perception of quality control in the company should emerge fairly 
precise expectations and goals for the audit. Such a careful preliminary orientation 
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Fig. 3 . Company Quality Audit 


of both parties ensures best the owner /manager ' s continued interest and active 
participation in the audit itself. The audit is built around the owner /manager 
entirely. A final audit plan shows various activities and its purpose and also a 
time schedule that allows full participation of client and auditor. An audit plan, 
formulated in writing constitutes a contract that can assist also in raising finan- 
cial support for the project. 

In order to guide further research and development in small company quality 
audits we designed an audit plan as a kind of tentative standard. If the 
owne r /manager requests a quality audit without the involvement in a preceding 
community audit, this company audit plan can still be applied. This standard audit 
for a small enterprise achieves the actual establishment of a practical and suitable 
quality program through the following stepwise approach: 

1. determine existing quality control/inspection practices. 

2. formulate these in writing in a simple language and augment with 
charts, etc. 

3. discuss, clarify and possibly revise these current practices with 
owner/managers and other individuals concerned. 

4. document approved practices as standard procedures in a quality manual. 

5. determine voids by comparing current practices with needs and possibly 
suitable generic standards for quality control programs. Have owner/ 
owner/manager decide whether additional control function and procedures 
are required. 

6. prepare simple forms for reporting and data compilation and document 
these in the manual. 

7. implement procedures and reporting schemes; possibly revise when 
necessary. 

8. prepare checklists and simple plans for self-audits to be conducted by 
the owner/manager or an employee and include these in the manual. 

9. set date for next self-audit. 

10. clarify future services by the auditor and enter auditor's endorsement 
of the quality program in the manual. 

These ten steps would have to be further extended in order to suit individual 
circumstances and personalities. 

The standard audit plan once jointly executed provides a most valuable learning 
experience for both partners. Owner /managers become adequately aquainted with modern 
quality assurance planning and with sound management practices. The focus on 
learning in an extremely practical and mutally beneficial manner strikes the main 
difference between this small business quality audit and others. Usefulness derives 



from the auditing exercise and not from a final report and recommendations. 

Moreover, the tangible outcome, the quality manual, represents and documents an 
operational and actually implemented and hopefully proven quality program. The 
owner/manager him/herself has prepared this manual and it constitutes most likely a 
real management breakthough. Without question, this program has every chance to be 
properly maintained through self-audits and improved when needs command this. The 
owner/ manager has become sufficiently trained in quality control directly in his 
company setting and at the same time has a quality program instituted with adequate 
evidence in form of the manual. Customers, suppliers, governments and otheis will 
share the benefit. 

One major problem in implementing these quality audits lies in the unavaila- 
bility of auditors. Further shared research and development by educators and 
practitioners will hopefully soon close the gap. 

SUMMARY 

This paper had to be confined to a general outline of our work in this project. 
Community and company quality audits together, once carried forward through coopera- 
tive efforts promise to create an invigoration of quality control at the local level. 
In our community we have found the concept and practice workable and useful. We help 
small business to help itself in building competitive strength through better quality 
control and ultimately to more satisfied customer and more productive business 
operations . 

While in our audit service, initiative rests and remains with the small business 
operator, leadership emanates from the local university and ASQC Section. It also 
allows government to play a meaningful guiding and supportive role in its small 
business policy and programs. 
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THE TRAINING & CERTIFICATION OF NUCLEAR 
Q . A. AUDITORS 

ALEXANDER STERNBERG 
PSE&G COMPANY 
NEWARK, NEW JERSEY 


INTRODUCTION 


Mr. Lee Everett, President of Philadelphia Electric Company at 
the ASQC Nuclear Quality Assurance Conference October 1-3, 1974 made 
the following statement: 

"Quality Assurance is at its best when it is doing an audit. The 
Quality Assurance Manager keeps all management informed about the 
information they need to know." 

Auditing is, if not the most important, at least one of the most 
important tasks performed by Quality Assurance personnel. Dating 
back to the earliest use of auditing as a financial tool, it has 
become management's mechanism for learning about what is actually 
taking place and the means for detecting potential problem areas as 
well as a source for preventing these problems from affecting cost, 
schedule, quality, and or safety aspects of the job. If used properly 
and on a timely basis it can be the most significant means for 
alerting management to all types of potential trouble areas thereby 
both minimizing their impact and establishing a media for their 
solution prior to catastrophic results. 

WHY AUDIT? 

The conducting of an audit involves clear and objective 
responsibilities on the part of all concerned. With the increasing 
use of qualification and certification, the recommendations and 
actions from audits are expected to have increasing importance. A 
properly conducted audit is a very important aid to management. For 
optimum value, audits should be properly organized for: purpose, 
coverage, schedule and planning. Proper organization eliminates 
confusion and delays. Planning makes for a more efficient audit 
and provides the proper professional approach. 

AUDITOR CERTIFICATION REQUIREMENTS 

It is not necessary to spend too much time discussing this topic 
since it is fairly well explained in ANSI Standards N45.2.12 and 
N45.2.23. I would, however, like to discuss this area a little 
further. Some other factors should be considered such as: 

1. A thorough knowledge of company policies and procedures. 

2. A knowledge of the applicable codes, standards, specifications, 
departmental or organizational procedures is necessary as 
related to the specific audit being performed. 

3. The ability to deal with people is a critical prerequisite. 
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QA AUDITOR TRAINING 


Auditors, in order to be most effective, must have certain traits 
A listing of those which are considered most important should help to 
illustrate this: Personable, inquisitive, systems oriented, objective 
good planner, problem solution directed, a good researcher, good 
speaker and writer, ability to cope with different personalities, 
knows how to develop and use the art of questioning, constructive, 
knows how to listen, and has the ability to discern "the vital few 
versus the trivial many" . 

When we consider the need for the above mentioned characteristics 
we conclude that the auditor must either initially have these traits 
or develop them via an extensive training program. This training 
program therefore consists of several parts. These parts occur at 
varying intervals starting with an individual undergoing orientation 
training upon joining the company, specialized auditor training while 
on assignment, follow-up training, and, when required, specialty 
oriented training covering such areas as NDE , Welding, special 
processes, etc. We will examine these in the order of their occur- 
rence. The order will be reviewed relative to time starting with 
preemployment requirements. 

1 . Pre-employment Requirements 


In order to facilitate a proper evaluation of prospective auditing 
personnel by the Personnel Department we have developed a 
personnel course, titled "QA Orientation For Personnel Department" 
An outline of this course is illustrated as Figure 1. The 
Personnel Department serves as the initial screener for' the 
basic traits required of those whom we intend to hire to perform 
auditing tasks. Obviously the objective of this stage is to make 
available those type individuals that are best qualified for 
development as auditors. 

2 . Orientation Training 

A new Quality Assurance Department employee is required to 
undergo a period of orientation training varying from 21 to 37 
days depending upon his past experience. During this period 
the orientee receives basic training preparing him for future 
auditing tasks. Some of the subjects covered are listed below: 

PSE&G QA Organization, Responsibilities and Role of 
Specific People 

Key Features of the QA Program 

QA Interface Relationships 

QA Requirements 

Codes and Standards 

Major Contractor QA Programs 

PSAR and FSAR 

Organizational Policies and Procedures 

Included within this training period is the requirement for the 
new employee to perform specific tasks such as: 

Review of a QA Manual 

Prepare audit questions based on the requirements of 
10CFR50, Appendix B. 

Participate in an audit 

Prepare for an audit including preparing an Audit Plan, 
conducting an Audit, and preparing an Audit Report 

Prepare a Procurement Control Plan 



The employee's progress is continually examined and evaluated. 
After the orientee satisfactorily completes this training he is 
assigned to a specific area within the Department. 

During the course of his regular activities on the job additional 
training is given as a part of individual technical development. 

This includes specific on-the-job auditor training, further class- 
room training, and specialty type training as required. On-the-job 
training involves further participation on an increasing responsi- 
bility basis for auditing. The individual's development is monitored 
by his supervisor. The program is developed by the department 
Training Engineer. Records are maintained on the individual and 
in the employee's personnal files. 

AUDITOR TRAINING 

If it is judged that more detailed training is required, the 
individual has to participate in a special auditor course. This 
consists of fifteen days of continuous training encompassing three 
days of classroom sessions and twelve days of closely monitored on- 
the-job training under the direction of the Training Group. The 
classroom sessions involve such topics as: 

Company QA Manual 
Preparation of Audit Questions 
Use of the CASE Questionnaire 
Elements of Control 
Objectives of QA Audits 
Type of Audits 

Essential Elements of an Audit System 

When to Audit 

Audit Frequency 

Qualifications of Auditors 

Preparation for Auditing 

Notification of an Audit 

Performing the Audit 

Post-Audit Conference 

Audit Reporting 

Audit Follow-up 

During the classroom sessions exercises are conducted using the 
technique of Role Playing presenting the trainee with simulated 
conditions including the various type personality traits he may 
meet during an audit and how he should handle each type. 

The training includes detailed discussions in such areas as 
Auditing Techniques, Questioning Techniques, Listening Techniques, the 
Use of Questions and Key Words for Questions. Included also is a 
review of the roles people play in meetings. Copies of some of the 
handouts distributed and discussed are attached as Figures 2-4. 

• To further develop the abilities of the prospective auditor 
additional courses presented by other departments within the company 
are made available. Two of these courses are Report Writing and 
Effective Speaking. Elements included in the Report Writing course 
are: 


General Principles 
Types of Reports 
Outlining the Report 
Composition 

Mechanical Form of Reports 


Application is included with the discussion of each topic 



The Effective Speaking course utilizes closed-circuit TV 
equipment. Each person must make several presentations of various 
content and length. Each presentation is filmed and then critiqued 
by all attendees. Elements of the Effective Speaking Course include: 

The need to speak effectively 
The personality of the speaker 
Presenting the talk 
Use of visual aids 

SPECIALTY AUDITOR TRAINING 

Specialty auditor training consists of specialized training 
covering such areas as NDE , Welding, ASME Codes, etc. In-house 
training courses, as well as courses given by outside consultants are 
used for these purposes developed and administered by the QA Training 
Group working with specialists in the applicable fields. 

Individual auditor proficiency is maintained by a continuing 
personnel evaluation program. Each individual's performance is 
regularly and annually appraised to assure proficiency is maintained. 

CERTIFICATION PROGRAM 

Certification of auditor is formally implemented utilizing the 
guidelines of ANSI Standards N45.2.12 and N45.2.23. Figure 5 is 
a sample copy of the form used and retained in the individual's 
training files. Great weight is placed on the individual ' s demon- 
stration of capability on the job, during annual performance 
evaluation. The certification program consists of two classifica- 
tions, Auditor and Lead Auditor. Minimum requirements involve the 
attainment of a minimum of ten points. 

Figure 5 illustrates the form used to document the qualifications 
for certification of an Auditor or Lead Auditor. The maximum 
number of points possible is 17, subdivided in accordance with the 
requirements of ANSI Standard N45.2.23 as follows: 

1. A maximum of 4 points with 2 for the equivalent of an 
Associate Degree to 4 for a Graduate Degree in specific 
disciplines . 

2. A maximum of 9 points based on experience in the field 
with emphasis on experience in nuclear QA Auditing. 

3. Two points for certification of competency issued by 
a state agency or national professional society. 

4. Two points for evaluation of competency by management. 

No one is certified as a Lead Auditor until total capability 
is demonstrated to cover all aspects of an audit including a 
thorough knowledge of the applicable requirements, awareness of 
personality traits, and other critical details so intimately 
concerned with auditing. 



TRAINING RECORDS 


The PSE&G QA Training Group was assigned the responsibility 
for maintaining the QA personnel training files. This includes 
copies of resumes, previous qualifications, eye examination results, 
attendance at workshops, conferences, courses, and symposia for each 
individual of the QA Department. Since this is a continuing require- 
ment, it requires follow-up. An example of this is the annual eye 
examination required. A tickler file is maintained alerting the 
training group to schedule an eye examination for a specific 
individual and to assure that this is accomplished. 

The records also include documentation of follow-up training. 
Figure 6 is a typical form used to document this information and 
illustrates the form used to document the periodic evaluation of 
personnel competency. Although the form is also used for evaluation 
in other areas, it serves as a measure to assure maintenance of 
proficiency in Auditing. The individual's performance in each 
auditing phase is evaluated, rated, and noted. This is reviewed 
annually with decisions made for further action based on the results. 
If inadequacy is discovered the individual undergoes additional 
training in his weak areas. 


SUMMARY 

A great deal more can be written about this subject. Discussing 
in greater detail such subjects as Key Control Tasks pertinent to a 
commercial nuclear generating station, the Quality Audit Cycle, 
Psychology of Auditing are also important elements which deserve 
examination. Books have been written on Auditing. This paper is 
just an introduction and I hope it opens the door to additional 
presentations discussing the Auditor rather than the Audit. It is 
certainly obvious that there can not be a successful audit without 
a technically competent and capable Auditor. How we assure that he 
is technically competent and capable is the key objective of this 
paper. Auditing, properly performed, can become the most significant 
tool toward assuring that projects are completed within established 
schedules. It can become the tool for reducing costs and preventing 
major problems. It can only accomplish this goal if implemented 
properly in a timely manner by technically competent personnel. 

BIBLIOGRAPHY 


R. Higgins, Armour Dial, Inc., "How To Listen & How To Ask Questions". 

Kachelek, R. L. "Quality Assurance Auditing", Quality , August 1977 
p. 19. 

Knight, F. W. "Auditing of Quality Assurance Programs for Nuclear 
Power Plants" 

Presented at ASQC 2nd Annual Middle Atlantic Nuclear Quality Assurance 
Seminar, March 4-5, 1974. 

MacKenzie, R. Alec. "The Time Trap", AMACOM, 1972, pp. 103-104. 

McCaskill Associates, "Personalities" Excerpt from unknown 
publication. Chapter XI. 

Vassalo, John R. "Psychology of Auditing", Quality Progress , 

September 1977, p. 37. 


LCS 320:90:549 



QA ORIENTATION FOE PERSONNEL DEPARTMENT 


1. Introduction 

1 . 1 Purpose 

1.2 Scope of QA Program 

1. 3 Definitions 

2. History of Nuclear Power Plant QA 

3. Nuclear Regulatory Commission 

4. Regulatory QA Requirements 

5. QA Department 

5.1 Organization 

5.2 Responsibilities 

5.3 Interface Relationships 

6. QA Personnel Qualifications 

6.1 Education 

6 . 2 Experience 

6 . 3 Training 

6.4 Personnel Characteristics 

6.5 Job Requirements 

6.5.1 Individual 

6.5.2 Interrelationships 

6.5.3 Scope of Job 

6.5*4 Vital Few vs. Trivial Many 


FIGURE 1 



KEY W ORDS FOR QUESTIONS 


CLASSIFY 

COMPARE 

CRITICIZE 

DEFINE 

DESCRIBE 

DISCUSS 

EXPLAIN 

ILLUSTRATE 

INTERPRET 

JUSTIFY 

OUTLINE 

REVIEW 

SUMMARIZE 

TRACE 

VERIFY 


Demands arrangement, assembling, or grouping of facts 
according to some common characteristic. 

Requires detection of the resemblance and the 
differences among facts. 

Exacts good judgement and careful analysis. 

Determine the boundaries of a subject and fixing of 
a clear meaning. 

Select and portray the characteristics of a subject. 

Present the pros and cons of a subject and come up 
with arguments supporting a position. 

Clarify points which obscure a subject. 

Calls for examples to clear up a discussion. 

Bring out the meaning of a subject as the person 
responding sees it. 

Show that a thing is reasonable or warranted. 

Indicate the main points of a subject. 

Compels going over a subject completely and giving 
it a critical examination. 

Ask for the presentation of a subject in a concise and 
compact manner. 

Follow in detail the development or progress of some 
sub j ect . 

Exacts proof that a thing is true. 


FIGURE 2 

THE USES OF QUESTIONS 


1 . 

To 

open discussion. 


2. 

To 

stimulate interest. 


3. 

To 

provoke thinking. 


4. 

To 

accumulate data. 


5. 

To 

get individual parti 

cipation . 

6. 

To 

develop subject matt 

er . 

7. 

To 

determine a member’s 

knowledge . 

8 . 

To 

change the trend of 

discussion. 

9. 

To 

arrive at a conclusi 

on . 

10. 

To 

terminate or limit a 

discussion 
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HOW TO LISTEN 


1. Remain natural. 

Do not give advice, agree or disagree, criticize or interrupt. 

2. Give your complete attention. 

Let him know you are listening. Nod you head -- "uh huh, I see 
what you mean." 

3. Ask about his statements. 

Dig out information, invite him to tell everything. Say: "l'n 
addition to that is there anything else...?" 

4. Restate his main points. 

Let him hear his exact words restated by you. This prompts him 
to stick to the facts and to think intelligently. 

5. Put his feelings into words. 

State what his feelings seem to be. When he hears them voiced 
by you he evaluates and tempers them. 

6. Get agreement. 

Summarize what you have both said -- encourage him to suggest 
the next step or course of action. 


HOW TO ASK QUESTIONS 


1. No third, degree. 

Use questions to help the other person think -- never to degrade 
or to spy. 

2. Asy "W" questions. 

What, Why, When, Where, Who and How are the key word s that will 
secure facts and information. 

3. Ask questions that make him go deeper. 

Ask for evidence, examples or explanat ions to discover reasons 
behind his thinking. 

4. Ask "suppose" questions. 

Introduce a new idea, break a deadlock or bring up an overlooked 
point with: "Suppose we...?" 

5 . Ask him. 

To encourage others to think or to avoid committing yourself, 
return the question or relay it to another qualified person. 

6. Ask questions that get agreement. 

Offer several solutions in the form of a question. 


Reprinted by permission of Mr. R. Higgins, Armour-l)ial , Inc. 
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QA AUDITOR PERSONNEL DATA QUESTIONNAIRE 


THE FOLLOWING INFORMATION IS NEEDED TO PROVIDE DATA FOR CERTIFICATION TO THE REQUIREMENTS OF QAI 2-7. PLEASE 
FILL IN AND RETURN TO QAD TRAINING ENGINEER. PLEASE INCLUDE A COPY OF YOUR RESUME. 

1.0 NAME: 

2 0 EDUCATION DATA: 

2.1 UNDERGRAD. DEGREE MAJOR YEAR 

NAME OF SCHOOL: 

ADVANCED DEGREES MAJOR .YEAR 

NAME OF SCHOOL 

3.0 EXPERIENCE DATA: 

3.1 YEARS EXPERIENCE IN ENGINEERING, MANUFACTURING, CONSTRUCTION, OPERATION 

OR MAINTENANCE 

3.2 YEARS EXPERIENCE IN NUCLEAR FIELD 

3.3 YEARS EXPERIENCE IN Q.A 

3.4 YEARS EXPERIENCE IN QA AUDITING 

3.5 YEARS EXPERIENCE IN NUCLEAR Q.A 

3.6 YEARS EXPERIENCE IN NUCLEAR QA AUDITING 

4.0 PROFESSIONAL COMPETENCE DATA: 

4.1 TYPE OF CERTIFICATION OF COMPETENCY IN ENGINEERING, SCIENCE OR Q.A. ISSUED AND 
APPROVED BY A STATE AGENCY OR NATIONAL PROFESSIONAL SOCIETY 


I CERTIFY THAT THE ABOVE IS TRUE AND CORRECT AND IN AGREEMENT WITH INFORMATION SUPPLIED BY 
ME TO PSE&G IN EMPLOYMENT APPLICATION (EXCEPT AS NECESSARY TO UPDATE THE INFORMATION). 

DATE: SIGNATURE OF EMPLOYEE 


5.0 RIGHTS OF MANAGEMENT DATA 

TOTAL POINTS 


FOR EVAL- 
UATOR USE 

POINTS 

ASGN. 

MAX. 

1 

4 


9 


2 


2 


n. 


DATE: .CERTIFIED AS: _ 

EVALUATED BY: 

REQUIREMENTS FOR LEAD AUDITOR: 10 POINTS MIN. 


FIGURE 5 


T R A I NEE EVALUATION 


Name of Trainee: 
Evaluator : 

Activity Involved : 

Date of Activity : 


Evaluation Checklist 
(Check Applicable Boxes) 
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THE CO-ORDINATION OF QA ASSESSMENTS AND AUDITS 


K.C. Ford, P. Eng. 
Canadian Standards Association 
Rexdale, Ontario, Canada 


Introduction 

With the advent of the nuclear power station, there has been a significant increase in activity 
relating to safety matters, by suppliers, users and governments. To design a unit that will function 
effectively and safely for 30, 40 or 50 years is not easy and in order to increase the level of 
assurance that the design is adequate, the various components are acceptable and the installation is 
correct, the quality control of yesteryear has been superseded by Quality Assurance requirements. 

In order to meet the various regulations which apply to nuclear applications and operations, it 
is now necessary for suppliers to have and operate a quality assurance program, to provide 
assurance that their product or service meets the requirements of the contract. Each purchaser is 
required to satisfy himself that the quality assurance operations of his suppliers are satisfactory. 
Because most suppliers are also purchasers, there is an additional need for those checks to also be 
carried out on their suppliers, and on their sub-suppliers... .to the point where either very basic 
materials are involved (such as a heat of steel where further, more basic, checks are not 
significant) or where the parts are not critical items. It is therefore easy to see how such a 
requirement can affect a large number of items, and therefore a large number of companies, 
because of this cascade effect. 

It is also typical that any one supplier sells to multiple purchasers; therefore such a supplier 
could be subjected to assessments by many, or all, of his customers. This is a problem, not only in 
duplication of work but also of differences in interpretation of the standards concerned. Herein 
lies both problem and opportunity. 

Standards 

In Canada we have a series of 4 standards; Z299.1, Quality Assurance Program Requirements, 
Z299.2, Quality Control Program Requirements, Z299.3, Quality Verification Program Require- 
ments and Z299.4, Inspection Program Requirements. 

The Canadian Nuclear Association originally prepared the drafts of these documents, and the 
scope was restricted to nuclear purchases. However, before being finalized, they were made the 
responsibility of a CSA Committee who broadened their scopes to be applicable to purchases by the 
electric power generation utilities. These standards were issued in 1975 as preliminary standards 
and are now being re-issued as full standards. 

The Canadian Standards Association (CSA) was established in 1919 and is a non-Government, 
not-for-profit Association whose corporate objective is Services in Support of Standardization. It 
has been accredited as a Standards Writing Organization (SWO) by the Standards Council of Canada. 
CSA has requested recognition in the subject area of the Assurance Sciences and intends to submit 
the Z299 series of standards for recognition as National Standards of Canada. 

The availability of 4 standards which delineate 4 levels of assurance means that a purchaser, 
having decided on the level required (depending on the criticality of the item) can then specify the 
assurance level required, clearly and easily. 

The principles given in these standards are applicable to almost all products and processes but 
additional education will be required before this is recognized, because many of those involved with 
mass-produced items tend to assume that the assurance sciences are for nuclear applications, must 
therefore be expensive, and are neither appropriate nor applicable to non-nuclear subject matter. 
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Background 

Suppliers to those utilities who are in nuclear power generation are required to meet the 
requirements of the relevant Z299 standard,- usually Z299.1 or Z299.2. Both of these standards 
require evaluation of subcontractors and because many suppliers purchase from multiple 
subcontractors - and they in turn often have multiple sub-subcontractors - there is a cascade 
effect. Many suppliers are finding that they are being subjected to multiple audits by their 
customers and also having to apply audits on their suppliers who are also receiving multiple audits 
from their customers! We recently heard of one large contractor who was involved with over 1000 
audits in a year - these included audits applied on him by his customers and by him on his 
subcontractors. 

Recognition of this duplication of effort and the basic unacceptability of expense without : a 
corresponding benefit has led a number of suppliers to try and find a means whereby the audit 
process could be made more efficient. Various trade associations as well as individual companies 
(suppliers, purchasers and users) have approached CSA to provide a co-ordinated management of 
these assessments and audits. 


Initial Steps 

Over this past year or so, we have had a number of requests for CSA to become involved with 
QA activities. Some have been from individual companies and some from trade groups. While most 
have been from producers, there have been enquiries from companies who supply services, and also 
a number of users, mostly utilities. 

Consideration was given to these numerous requests and a number of small, often informal, 
meetings were held in order to ascertain what the prevailing situation might be and whether there 
was a real need. On past occasions we have been approached and asked to provide a program which 
has had only partisan support and we wished to assure ourselves that there was a real need before 
spending any significant amount of time and effort on the subject. 

These discussions have satisfied us that a program for the management and consideration of 
assessments and audits would fill a real need, would be widely supported and - a not insignificant 
point - would be cost-effective. Although the preparation and installation of a QA program costs 
money in the initial stages, it is an investment and should be considered in the same way as any 
other business investment. Various companies have provided data to show that there is a 
significant saving to the overall quality costs when an appropriate QA program is installed. The 
economic arguments for the installation of a QA program are among the most persuasive ones. 

Having satisfied ourselves that there was a valid need, we had to consider whether CSA could 
fill that need. The requirements appear to call for an independent organization which was national 
in its operations and was also widely recognized. It was also considered necessary for the 
organization to be able to do the job well - a poorly executed program would be & disservice to all 
concerned. 


Because of our past experiences with product certification we have considerable experience 
in certain technical areas and, more importantly, many years of experience in managing 
independent, third-party programs for various manufacturing industries and our programs have been 
widely recognized by both the Authorities and purchasers. Because we have a considerable 
background in both standards writing and in the interpretation and application of standards to 
practical situations, to manage and co-ordinate an audit program appears to be a logical extension 
of our current activities. 


The next step was to have detailed discussions with one trade association whose members had 
expressed an interest in CSA managing a QA program. After several meetings, first with their 
technical and later with their management representatives, it was agreed that we would run a pilot 
project with their members. This is now being carried out and is providing CSA with additional 
information and background relating to the operation of such a program. The participating 
companies are gaining the benefit of an independent appraisal of their documentation and 
operations. 


Factory Assessment 

One of the first steps required when executing a facility assessment is to know which level 
the company intends to work to, and to then review their QA manual against the relevant standard. 
This is carried out before any assessment of the actual operations is made. 

It is preferable that the manual reflects what is actually done in the plant, rather than be a 
theoretical^document based on the requirements of the standard Many o the current operatmns 
and techniques already in use in the plant are often acceptable. It is usually easier for the manua 
to reflect what is done and then upgrade procedures and practices as necessary by a process of 
evolution, rather than to try and impose a theoretical document on the operation. 

Durinff this review it is necessary to decide on a number of points: is the material required by 
the standard ind.uded? W if not, it must be added: is the material relevant to the requiremen s of 
the standard 9 if not, it can be ignored - provided it is not in conflict with any other parts of the 
manual ft it is relevant, then it to clear, complete and unambiguous? if not .t needs rev.s.on lt 
can therefore be readily seen that, in most cases, the preparation of a manual will be a reiterative 
process. 

It should be noted that, although the review of the manual is easier when the manual sequence 
corresponds to that of the standard, it is preferable for the manual to be so arranged that those 
who will need to use it find it easy to use. 


Although it is desirable to have a complete and acceptable manual available before a factory 
assessment begins, this is not always practical. For example, a review of a manual may show it has 
a number of points needing change, correction, expansion, or clarification. While one can wait until 
the manual is revised, and the subsequent review shows it to be acceptable, there are advantages to 
making a partial factory assessment based on the available manual: if the manual needs further 
work it is probable that there will also need to be some revised operating procedures and 
instructions, and a partial assessment enables progress to be made on both documentation and 
operations concurrently. 

Although having a complete and acceptable manual is important, it is even more important 
for the operations to produce the correct results: paper is not a substitute for quality. It is during 
operations that the required quality is built in; the associated paperwork is only a means to the end, 
not the end itself. 

Assessment Teams 

The assessment of the facility is probably the most important part of the whole program. In 
order that it be executed effectively, it is planned that each team be specifically selected to 
include adequate and appropriate expertise for the technology involved during the assessment. This 
means that the team will typically include consultants having the specific expertise. They will be 
under the general overall control and guidance of the CSA staff man who will be both the team 
leader and the senior auditor. 

The team leader will be responsible for the overall assessment including the management 
interviews, manual reviews, the direction and organization of the assessment of the facility itself, 
as well as the subsequent feedback and follow-ups as necessary. 

Records 

CSA will also act as the repository for the assessment records - the consolidated information 
covering manual review, the assessments and follow-ups, subsequent audits and the related records. 
The publication of a list of those companies who have successfully met the standard will be issued 
on a regular basis and will include the company identification, the level to which they have been 
assessed, and the product range or group covered. This list will be a Register of Companies having 
Assessed Capability and will be of considerable help to purchasers when selecting suppliers, because 
they will then have the benefit of an independent assessment without the cost of having to perform 
it. 

Qualification Program 

The initial work on assessments has been done in a limited sphere, but based on the data 
generated, the CSA Board of Directors have taken steps to move the pilot project toward a full 
program, to be phased in, on a sector by sector basis. A Policy Board is being formed by the 
Association, to provide the overall guidance to,CSA activities in this field. 
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The scope of the program is potentially very wide because it appears to offer benefits to all 
sectors of the manufacturing and processing industries as well as to many service organizations. It 
is planned that the program be introduced one industry or sector or group at a time, as staff 
capability is expanded. It is considered preferable for smaller segments of industry to be activated 
and the initial work completed relatively quickly, rather than to offer a wide-ranging program 
initially which cannot then be serviced on a timely basis. 

The rate of progress will be partly dependent on the availability of assessment teams, but will 
depend more on the amount of activity a company applies to having their program operating 
completely and effectively. The team has the responsibility of assessing that which is happening; 
the company has the responsibility of making it happen. 

Having a single organization responsible for the management of assessment and audits offers 
further benefits to the program because any problem areas will be treated equitably and the 
interpretations given will be uniform. A program of this nature must not only be equitable but 
must be seen to be equitable. 

Program Acceptance 

A coordinated program, such as outlined above, can only be considered fully successful and 
provide maximum benefits when it is accepted by effectively all those purchasers, users and 
authorities who would otherwise need to perform the work themselves. To this end, we are working 
to get the program known and understood. 

We have had some discussions, and more are planned, with large purchasers - who are also the 
final owners in many cases - and with Authorities so that they are aware of what the program can 
offer and also to acquaint them with what the program will not do for them. There will be further 
discussions with various trade groups and associations to ensure that there is adequate two-way 
communication with those who are actively involved, and these are expected to take place as we 
are able to offer the program in their product areas. 

As noted earlier, the technical side of the program must be complete and effective, because 
otherwise the whole activity becomes superficial and offers false security at best. Executed 
properly, it offers the advantages of a coordinated, standardized approach to the uniform 
qualification of QA systems, without duplication of effort. 

Conclusion 

Because the introduction of a quality assurance program into a supplier's operation seems to 
consistently pay dividends, there are direct benefits to the supplier. When a purchaser buys from a 
supplier who successfully operates a quality assurance program he has a greater assurance that he 
will receive what he needs and will also typically receive it on the agreed schedule. Similarly, the 
Authority having Jurisdiction can have a greater confidence that both the spirit as well as the 
letter of the law is being met. 

This program is not intended to reinvent the wheel, nor to duplicate what is already being 
carried out by others but to coordinate the various efforts now being expended. It follows naturally 
from our corporate objective - Services in Support of Standardization. 

This is a most exciting time for those of use who are involved with QA because we are on the 
threshold of a program which can offer significant benefits to all - supplier, purchaser, company 
and country. 
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EXTENDING EFFECTIVENESS OF QUALITY COST PROGRAMS 


Richard K. Dobbins, Principal Quality Assurance Staff Engineer 
Honeywell, Inc. - Process Control Division/Fort Washington, Pa. 


EVOLUTION OF QUALITY COST PROGRAMS 

Does your company have a quality cost program? If so, would it be 
classified as active or passive ? What primary purpose did the program 
originally serve? What purpose does it serve now? And, most important- 
ly, what reaction does it generate within your executive offices? 

The answers to questions like these would prove very interesting, 
if it were possible to obtain factual replies from all American business 
concerns with more than 500 employees. While no valid studies are 
available, the A.S.Q.C. Quality Cost Technical Committee has reason to 
believe the following relationships would probably apply: 

1 . The majority of companies still do not use any formalized 

quality cost program . 

o Most manufacturing companies regularly monitor certain key 
quality cost elements, such as scrap, rework. Inspection 
budget costs, etc. 

o Other management techniques are frequently employed, which 
may bring considerable temporary cost consciousness to part- 
icular products or departments. Examples include: 

A. Budget controls and restrictions. 

B. Value Engineering or cost reduction studies. 

C. Cost improvement goals within Management By Object- 
ives . 

D. Employee suggestion systems. 

E. Management bonuses, incentive systems, and profit 
sharing . 

o Formal quality cost programs are quite rare within Service 
Industries, or companies having high service-to-product 
ratios . 

2 . Use of formalized quality cost programs is steadily growing 

within Manufacturing Industries, and there is a noticeable 

quickening in growth rate . 

o In 1967, when the first edition of " Quality Costs - What & 
How " was published, quality cost analysis was mainly an 
unknown academic curiosity within technical circles of the 
quality profession. 

o In 1971, when the expanded second edition of this manual was 
published, these techniques had been accepted in principle 
by only the most progressive of Manufacturing Industries, 
and experimental programs had begun. 

o By 1977, when " Guide For Reducing Quality Costs " was pub- 
lished, routine use of quality cost analysis was an accepted 
practice for a significant minority of manufacturing comp- 
anies. Still others were just instituting trial programs. 

o Within the past five years, interest levels have been pro- 
gressively increasing, as indicated by attendance at the 
tutorial and management sessions at the Annual Technical 
Conferences, regional, divisional and section symposiums. 
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Initial applications of formal quality cost programs are 
usual ly directed towards specific pre-existing malignancies 
of major proportions . 

o Quality cost techniques are used to identify and focus 
attention on these known aggravated problems, (generally 
product lines or manufacturing processes) . 

o After quantification, priority managerial approval is ob- 
tained to mount a coordinated attack by all concerned 
parties . 

o Cost analysis reports document the early economic results of 
this campaign, until all proposed corrective actions have 
been implemented, (or other crises require reduction in 
attack effort) . 

o Subsequent reports document the longer term impact of the 
earlier task team corrective actions, and soon reports be- 
come very routine until a new crisis causes the cycle to be 
repeated . 

4 . The normal use of a quality cost program may, therefore , 

deteriorate into a "scorecard" posting, with little sustaining 
executive influence . 

o Reports serve chiefly as a reminder of previous "glorious 
accomplishements" , plus a general reassurance that present 
day quality is still "in control". 

o Complacency is further promoted through favorable comparison 
with : 

A. Non-exact general industry approximations. 

B. Past company history before any quality cost program. 

C. Conservative or "safe" company goals. 

o Most quality cost programs are maintained only on a compos- 
ite company or divisional basis, without ready segregation 
of specific product lines or manufacturing processes. 

o Unless violent unfavorable events occur, little effect may 
be indicated in the overall "scorecard" rating. 

o When used only in a passive manner, the analysis indicates 
results which have already occurred, with no opportunity to 
change what is now past hastory. This further increases the 
"scorecard syndrone" on the part of executive reaction! 

QUALITY COST ANALYSIS - SO WHAT? 

Unless dynamic response is introduced into your quality cost pro- 
gram, it may be found that your company's reaction will have rotated 
full circle from " Quality Cost Analysis - What's That? " to " Quality Cost 
Analysis - SO WHAT? " Companies should not use a quality cost program 
simply for the sake of using one, or because they believe they have a 
better quality cost rating than the average company in their line. And 
they certainly shouldn't start one because best Q.A. theory indicates it 
is the right way to do things. It is much more important to "do the 
right things" than it is to "do things right"! 

DOING THE RIGHT THINGS 

From an executive management viewpoint, what type of "right things" 
need to be done in which Quality Assurance can exert a positive influ- 
ence? Examples include: 

o Early identification of problems, with assessment of magnitude 
and trend patterns. 

o Establishment of priorities for efficient utilization of avail- 
able limited resources, and decision making. 

o Administrative structuring to have middle management exercise 
active responsibility, within their authority, for total company 



interests . 


0 Combat complacency and inertia through actions to improve the 
status quo. 

A well designed quality cost program can be an effective management 
control tool, instead of simply a "scorecard" of composite quality cost 
attainment. But, how can this happy state be achieved? 

An active quality cost program will assist and permit the manage - 
ment of total quality costs consistent with company policy. But, to 
"manage", we must control . to "control", we must measure . To "measure", 
we must define . In "defining" , we must quantify . Keeping these ele- 
mentary requirements in mind, the following guidelines are presented for 
construction of a powerful quality cost program. 

ANATOMY OF A DYNAMIC QUALITY COST PROGRAM 

1 - Provide for detailed cost identification and segregation down 

to the lowest level where control is desired . 

Remember, in combined form, the individual elements are 
not capable of measurement, and therefore can't be controlled. 
It may not be necessary to segregate each indiviual product 
model, but general product lines or families should be separat- 
ed from other costs. Within manufacturing facilities, consider 
whether "machine shop" might be too broad a listing, and yet 
"machine lathes", "turret lathes", "multi-spindle", and "auto- 
matic screw machines" might all be combined into "lathe family". 

2 . Reflect all identified factory or field quality costs back to 
the vertical lines responsible for the products and services 
involved . This includes allocating the entire Quality Depart- 
ment budget i 

Don't lump any quality cost elements into big, nondescript 
collections, such as "field service costs", or "general scrap". 
Should the Field Service Manager be held accountable for the 
costs associated with servicing field products which become 
defective during the warranty period, or should the responsible 
Vertical Line Manager? Should not that responsibility include 
all factory and field costs identified with that product or 
service? If so, then the Q.A. and Inspection costs attributed 
to the support of that product (both in the factory and in the 
field) should also be allocated back to that Vertical Product 
Line. Generally speaking, all identified or allocated costs 
should reflect back to the same organizational groups which 
received credit for the units of production, and net sales 
billed. 

3 . Establish bold cost improvement objective goals, which are 
believed to be achievable, but only through sustained, hard 
effort . 

Don't establish next year's goals based upon actual costs 
for this year, plus an allowance for inflation. This keeps 
increasing management's "threshold of pain" to the point of 
insensitivity to all but major calamities. Long established 
costs do not necessarily mean unavoidable costs . Challenge 
them ! 

4 . The cost improvement objectives established should be incorpo - 
rated as personal goals of the lowest managers who have overall 
discretionary authority for the product or service involved . 

Individual objectives for the product lines or services 
should then be combined as composite objectives upwards within 
the chain of command, in the same manner as line responsibil- 
ities are combined upwards. How else can you require these 
managers to control their quality costs in the same manner they 
now control their direct and indirect costs, and manage their 
production schedules? 



5 . 


Consider all high quality cost elements as being potential cost 
breakthrough opportunities . 


Use Pareto analysis and trend analysis techniques to ident- 
ify such opportunities for concentrated attack through design, 
material, process, methods and equipment challenges. Initiate 
Value Engineering studies, and invite suggestions from the 
people who work on the product line or in the processing area 
involved. Compare how your competitiors handle that aspect 
within their products. Consider new technology which has been 
introduced since your design or process method was first imple- 
mented . 

6 • Incorporate specific quality cost objectives into all new or 
improved product design efforts, or services offered . 

Quality cost objectives should be just as important as for- 
mal estimates of direct production costs, which are normally a 
specified criteria of every new release. (Also, don't forget 
that MTBF goals should be included in all Sales-Engineering 
project specifications for new releases.) In the same manner as 
the direct cost estimates, quality cost objectives should be 
agreed to by the Vertical Line Manager who will finally be re- 
sponsible for the new release. Until achieved or renegotiated. 
Engineering should be held jointly responsible with the Vertical 
Line Manager for excessive direct or quality costs associated 
with the new product release. 

7 . Require rapid collection of current data, and perform timely 
trend analysis using appropriate multiple bases . 

The object of this exercise is the creation of an "early 
warning system", to permit necessary interim, adaptive and 
corrective actions. Rapid actions will prevent or limit addit- 
ional avoidable losses from being sustained. Doing this will 
provide the dynamic response needed to overcome the historical 
"scorecard syndrone". 

8 . Use the quality cost program to manage the available Quality 
Assurance budget and resources where they will be most effective . 

Whether your departmental budget and resources are generous 
or meager, stop doing other people's work for them , and concen- 
trate on your own major quality responsibilities. As Quality 
Manager, you are not responsible to cure every problem discover- 
ed! But you are responsible to see that each such problem is 
properly addressed by the appropriate action parties, and then 
satisfactorily resolved! 

BENEFITS OF A DYNAMIC QUALITY COST PROGRAM 

Quality Assurance should intensify efforts towards the identificat- 
ion of existing or potential problems, assessing their magnitude and 
impact, referral to the proper parties for resolution, and monitoring 
corrective action efforts and effectiveness of same. In all these 
endeavors, a dynamic quality cost program could be the vehicle to manage 
the majority of these efforts. 

Such a quality cost program would then be truly an effective manage- 
ment control tool, and not just a historical "scorecard" grade. Just as 
a machine tool provides a distinctive mechanical advantage, so this 
diagnostic control tool provides a distinctive "managerial advantage". 

This advantage can then be directed towards more effective management 
decisions, increased profitability, better return on investment, improved 
utilization of Quality Department resources, and elimination or improve- 
ment of product lines or services with marginal future economic worth. 
Isn’t this a giant-sized improvement over former intuitive or stereotyped 
management methods? It can assure a progressive company of increased 
profitability and cost effectiveness, and permit it to engage in keener 
competition with smaller risks. That, in turn, can mean greater market 
share, whether times are good or bad. 



THE QUALITY ASSURANCE CHALLENGE 


Have you offered the executive management of your company a dynamic 
quality cost program yet? Consider how your competitive position might 
now have been improved, if. such a program had been implemented two years 
ago! Can you afford to wait until after your major competitor has one in 
operation? 
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"A METHOD FOR PREDICTING WARRANTY COSTS 1 ' 

Andrew F. Grimm, Manager, Quality Assurance 
Clark Equipment Company - Axle Division, Buchanan, Michigan 


Data generated in a reliability evaluation of a product can be an important start- 
ing point for predicting potential costs involved with product failure at any future 
point in time. A conversion of failure rates into cost data can prove to be a valuable 
information contribution to business managers in forming decisions about continuation 
of a product program or at least provision of initial and ongoing capital budgeting 
plans for a potential product's production and introduction to its market. 

The objective of this paper is to illustrate a method for translating a product's 
reliability profile and design alternative considerations into a cost structure that 
can provide business management with information they can understand. The extra steps 
of this method could prove worthwhile in simplifying illustration of the underlying 
potential warranty problems and associated costs that might not be truly understandable 
in its original technical form. 

Basically, the three parts of the method are the original reliability evaluation 
of a design and its alternatives, the application of Markov Chain Theory in extending 
failure rate data into the future, and the analysis of applied costs to this point by 
the net present value method in order to determine project profitability potentials 
between the original design and its alternatives. Markov Chain construction can provide 
information about summation of product failures at future points in time as production 
continues and the population of product in use increases. This profile provides info- 
rmation about product in use accumulation distribution that is necessary for providing 
a base to which design failure rate information can be applied in predicting the prob- 
able quantities rate of return of failed goods. Of course, return of goods may be cov- 
ered under both implied product warranty as well as stated warranty policy. 

In order to explain the method, an actual example has been selected to illustrate 
the steps taken in arriving at project cost information that will be in a simplified 
form that business managers can find to be usable in forming decisions about a project's 
viability and continuation. Although this project example is real, certain elements 
have been changed so as to prevent recognition of the actual product. This author 
believes that the changes that have been made to prevent recognition of the product do 
not materially affect the outcomes contained in the example's illustrations. The exam- 
ple selected is a small analog computer, that has a fixed program for controlling a mec- 
hanical function of some machinery. Knowing the actual controlling outputs of this dev- 
ice are not necessary to understanding the evaluation of the reliability of the product. 
However, let us end this product's application with the note that it is an integral part 
of a higher order system. The product part we will use in our illustration is not main- 
tainable, but is replacable in the overall system. 

The computer is composed of a variety of electronic components mounted on three 
stacked PC boards. These electronic components consist of the usual varieties includ- 
ing transistors, pots, capacitors, integrated circuits, diodes, resistors, etc. Our 
study will be confined to the transistors, ICs and diodes involved in the design. The 
original design contains the following quantities of each component under study: IC's -2, 
Transistors - 14, Silicon Diodes - 25. 

Reliability information about these components is drawn from MIL-HDBK-217B, "Rel- 
iability Prediction of Electronic Equipment". Table 1. contains a summary of this info- 
rmation as used in this study. Data was generated for components operating at 25° C 
ambient and at two levels of quality, ie: commercial vs military, (Jan). Table 1. con- 
tains the pertinent information concerning the design alternatives under consideration. 
This information includes the combinations of commercial versus military quality comp- 
onents for each design alternative which in turn influence the total unit failure rate. 
Of course, the total unit incorporates other components which for simplicity in this 
example are considered held constant. Additional information contained in Table 1, is 
the cost to build per unit which takes into account the added cost of "burn in" and sort 
testing of military quality components. Comparing the failure rate column with cost to 
build does not clearly define a relationship as to which combination delivers the opt- 
imal reliability to cost relation. Thus the business decision becomes quite difficult 
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unless additional means of analysis is found to overcome the indecisiveness of this 
problem. The method we will propose is one means of bridging the current state of our 


problem. 



TABLE 1 
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Failure 

Cost to 

Alternative 

12 

Diode 

Transitor 

Rate % 

Bui 1 d/unit 

1 

c 

c 

c 

12.6 

$30.00 

2 

m 

c 

c 

5.9 

30.14 

3 

m 

m 

c 

3.8 

31.89 

4 

m 

c 

m 

4.6 

31.54 

5 

c 

m 

c 

10.5 

31.75 

6 

c 

m 

m 

9.3 

33.15 

7 

c 

c 

m 

11.2 

31.40 

8 

m 

m 

m 

2.5 

33.29 



m - mi 1 i ta ry , 

(Oan) 

c - commercial 



Let us assume that we must generate a returned goods rate based on the failure 
rates indicated in Table 1, for each component design quality level. The method used is 
the Markov chain. Application of this technique will develop the number of units exp- 
ected to fail within each time segment of the warranty period. Our example will utilize 
a five year warranty period. Instead of developing Markov chains for each of our eight 
design alternatives, we shall only develop an example of a Markov chain for a 10% fail- 
ure rate over a 5 year warranty period. Figure 1 is the graphical illustration of this 
example Markov chain. 


FIGURE 1 


Markov Chain for: Failure Rate, 10%; Warranty Period, 5 years. 
Period (yr.) 1_ 2_ 3_ 2 2 



Line Code: 


90% Success Units 


10% Failure Units 


Note that failures are replaced with additional production in the following time segment 


and that those replacement units are subject to the same failure rate in each time seg- 
ment. The Markov chain is an excellent tool to handle this condition for production 
replacement. Table 2 is a summation of the Markov chains for each design alternative 
in the example. 

TABLE 2 


Design 

Sale 


Warranty Year 



Alternative 

Type 

1 

2 

3 

4 

5 

1 . 

Original 

100,000 

111,012 

122,024 

133,036 

1447048 


Replacement 

12,600 

14,188 

15,776 

17,364 

18,952 


Total Prod. 

112,600 

125,200 

137,800 

150,400 

163,000 

2. 

Original 

100,000 

105,552 

111,104 

116,656 

122,208 


Replacement 

5,900 

6,248 

6,546 

6,944 

7,292 


Total Prod. 

105,900 

111,800 

117,650 

123,600 

129,500 

3. 

Original 

100,000 

103,656 

107,311 

110,967 

114,622 


Replacement 

3,800 

3,944 

4,089 

4,233 

4,378 


Total Prod. 

103,800 

107,600 

111,400 

115,200 

119,000 

4. 

Original 

100,000 

104,388 

108,777 

113,165 

117,554 


Replacement 

4,600 

4,812 

5,023 

5,235 

5,446 


Total Prod. 

104,600 

109,200 

113,800 

118,400 

123,000 

5. 

Original 

100,000 

109,398 

118,795 

128,193 

137,590 


Replacement 

10,500 

11,603 

12,705 

13,808 

14,710 


Total Prod. 

110,500 

121,001 

131,500 

142,001 

152,500 

6. 

Original 

100,000 

108,435 

116,870 

125,305 

133,740 


Replacement 

9,300 

10,165 

11,030 

11,895 

12,760 


Total Prod. 

109,300 

118,600 

127,900 

137,200 

146,500 

7. 

Ori ginal 

100,000 

109,946 

119,891 

129,837 

139,782 


Replacement 

11,200 

12,454 

13,709 

14,963 

16,218 


Total Prod. 

111,200 

122,400 

133,600 

144,800 

156,000 

8. 

Original 

100,000 

102,437 

104,875 

107,313 

109,750 


Replacement 

2,500 

2,563 

2,625 

2,688 

2,750 


Total Prod. 

102,500 

105,000 

107,500 

110,001 

112,500 


Combining information from tables 1 and 2 and figure 1 allows development of info- 
rmation for tables 3 and 4. (Note: Due to the size of tables 3 and 4, they are contai- 
ned in Appendix b)« Table 3 is the business information matrix leading to the NPV of 
table 4. A discussion of each element in Table 3 will allow an understanding of each 
idea. The design alternative is taken from Table 1 and indicates the mix of commercial 
quality to military quality in the three components. The year column signifies each 
year in the warranty period. Units for sale is taken from Original Sale Type in Table 
2. Cash In Flow at $50 is the total income of units for sale at a price of $50 per unit 
Units Built is the sum of original production and replacement production based on Markov 
chain outcomes at each design alternative failure rate. Cost to Build is the product 
of Units Built times the cost of manufacture. Again, unit cost of manufacture is based 
on the mix of military to commercial quality of the three basic electronic components we 
are studying in our example. Units Returned Warranty is the sum per period of failures 
calculated in the Markov chain. Additional Cost @ $25 Warranty is the product of Units 
Returned times the cost of handling warranty that is additional to the production cost 
shown in the Cost to Build column. Cash Outflow is the difference between the column, 
Cash Inflow @ $50, and the summation of the columns. Cost to Build plus Additional Costs 
0 $25 Warranty. 

Table 4 represents the steps in calculating the net present value for each of the 
design alternatives. This is the business decision matrix. The first column is Cash 
Outflow. This column is the ending column in Table 3. The next column is Depreciation 
of capital assets. For simplicity, we have selected the straight line depreciation 
method for a five year period. Net Income B4 Taxes is the difference between Cash In- 
flow and the summation of Cash Outflow plus Depreciation. The next column. Net Income 
After Taxes is based on a corporate tax rate of 48%. (Note: The corporate tax rate may 
vary according to changes in tax law and may be subject to change. This means that the 
reader should check the current corporate tax rate before using the method.) Net Cash 
Flow shows the first figure as a negative number. This figure is the cost of capital 
for the project term, in our case 5 years. The negative figure is also the sum of the 
Depreciation column for the project term. The remaining figures in Net Cash Flow are 
the difference between Net Income B A Taxes and Net Income After Taxes. The second from 
the last column is the rate of discount. This is the rate available to borrowers of 
capital on the money market at the time the project is being considered. The final col- 
umn, NPV is the product of the Net Cash Flow times the value of the discount rate in 
that year of the project. Discount rate values are usually obtainable from discount 



tables contained in many texts in Accounting, Finance, Managerial Economics, etc. The 
project with the highest project NPV is the most desirable business alternative provided 
assumptions we have made hold true for the project term. Limiting assumptions are; cost 
of capital is correctly defined; corporate tax rates do not change; discount rates will 
hold fairly steady; etc. 


Design 

Alt. 

Fail ure 
Rate (%) 

Production 

Cost 

Per Unit 

TABLE 5 

5 Yr. Total 

Net Income 
After Taxes 

5 Yr. Total 
Sales 

5 Yr. Period 
Net Present 
Val ue 

1 

12.6 

$30.00 

$4,004,000 

$30,506,000 

$3,044,128 

2 

5.9 

30.14 

4,688,770 

27,776,000 

3,596,890 

3 

3.8 

31.89 

4,344,064 

26,827,810 

3,338,260 

4 

4.6 

31.54 

4,378,421 

27,194,200 

3,361,391 

5 

10.5 

31.75 

3,658,014 

29,698,800 

2,783,744 

6 

9.3 

33.15 

3,348,448 

29,217,500 

2,547,167 

7 

11.2 

31.40 

3,683,680 

29,972,800 

2,802,026 

8 

2.5 

33.29 

4,054,540 

26,218,750 

3,118,676 


Table 5 summarizes pertinent information from the myriad of calculations in previ- 
ous tables. The first three columns have already been explained, but are included to 
define individual project identities. Column, 5 Yr. Total Net Income is the sum of the 
project term figures from the Net Income column in Table 4. 5 Yr. Total Sales is the 

sum of the Cash Inflow column in Table 3, and 5 Yr. Period Net Present Value is Total 
NPV in Table 4. 

Analysing the results indicates that from a business decision viewpoint, design 
alternative 2 is the most feasible, relating to NPV. It is also most feasible when 
considering Net Income After Taxes. However, it is not the most powerful when solely 
considering total sales. It isn't too bad when evaluating unit production cost. 

From the Engineering viewpoint, design alternative 8 would be most feasible since 
it indicates the lowest failure rate. If the product in fact does require high reliab- 
ility considerations in use, note that the NPV is fourth best. The balance between 
technological and business evaluation is fairly good. The value of this method gives 
information that can allow a less uncertain environment in which a decision can be made. 

The method outlined attempts to apply business decision techniques to technological 
evaluation in order to create a less risky environment in which project selection must 
be conducted. Cost information applied to technological reasoning rounds out the 
manager's decision making responsibil ity toward a higher level of certainty. If the 
limiting assumptions do in fact hold true for the period under study, we find that this 
method can actually allow for control of the amount of warranty returns and associated 
costs in each period by selecting the design alternative most appropriate to the busin- 
ess and technical environment conditions and criteria. 
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APPENDIX B 


Table 3 


Des. 


Uni ts 

Cash 

In Flow 

Units 

Cost To 

Warranty 

Units 

Warranty 
Costs at 

Cash Out 

Alt. 

Yr. 

For Sale 

At $50 

Built 

Build 

Returned 

$25/Unit 

Flow 

1 

1 

100000 

$5000000 

112600 

0 $30.00 
$3378000 

12600 

$315000 

$3693000 


2 

111012 

5550600 

125200 

3756000 

14188 

354700' 

4110700 


3 

122024 

6101200 

137800 

4134000 

15776 

394400 

4528400 


4 

133036 

6651800 

150400 

4512000 

17364 

434100 

4946100 


5 

144048 

7202400 

163000 

4890000 

18952 

473800 

5363800 

2 

1 

100000 

5000000 

105900 

@ $30.14 
3191826 

5900 

147500 

3339326 


2 

105552 

5277600 

111800 

3369652 

6248 

156200 

3525852 


3 

111104 

5555200 

117650 

3545971 

6546 

163650 

3709621 


4 

116656 

5832800 

123600 

3725304 

6944 

173600 

3898904 


5 

122208 

6110400 

129500 

3903130 

7292 

182300 

4085430 

3 

1 

100000 

5000000 

103800 

0 $31.89 
3310182 

3800 

95000 

3405182 


2 

103656 

5182800 

107600 

3431364 

3944 

98600 

3529964 


3 

107311 

5365550 

111400 

3552546 

4089 

102225 

3654771 


4 

110967 

5548350 

115200 

3673728 

4233 

105825 

3779553 


5 

114622 

5731110 

119000 

3794910 

4378 

109450' 

3904360 

4 

1 

100000 

5000000 

104600 

0 $31.54 
3299084 

4600 

115000 

3414084 


2 

104388 

5219400 

109200 

3444168 

4812 

120300' 

3564468 


3 

108777 

5438850 

113800 

3589252 

5023 

125575 

3714827 


4 

113165 

5658250 

118400 

3734336 

5235 

130875 

3865211 


5 

117554 

5877700 

123000 

3879420 

5446 

136150 

4015570 

5 

1 

100000 

5000000 

110500 

0 $31.75 
3508375 

10500 

262500 

3770875 


2 

109398 

5469900 

121001 

3841782 

11603 

290075' 

4131856 


3 

118795 

5939750 

131500 

4175125 

12705 

317625 

4492750 


4 

128193 

6409650 

142001 

4508849 

13808 

345200 

4854049 


5 

137590 

6879500 

152500 

4841875 

14910 

372750 

5214625 

6 

1 

100000 

5000000 

109300 

0 $33.15 
3623295 

9300 

232500 

3855795 


2 

108435 

5421750 

118600 

3931590 

10165 

254125 

4185715 


3 

116870 

5843500 

127900 

4239885 

11030 

275750 

4515635 


4 

125305 

6265250 

137200 

4548180 

11895 

297375' 

4845555 


5 

133740 

6687000 

146500 

4856475 

12760 

319000 

5175475 

7 

1 

100000 

5000000 

111200 

0 $31.40 
3491680 

11200 

280000 

3771680 


2 

109946 

5497300 

122400 

3843360 

12454 

311350 

4154710 


3 

119891 

5994550 

133600 

4195040 

13709 

342725' 

4537765 


4 

129837 

6491850 

144800 

4546720 

14963 

374075 

4920795 


5 

139782 

6989100 

156000 

4898400 

16218 

405450 

5303850 

8 

1 

100000 

5000000 

102500 

0 $33.29 
3412225 

2500 

62500 

3474725 


2 

102437 

5121850 

105000 

3495450 

2563 

64075 

3559525 


3 

104875 

5243750 

107500 

3578675 

2625 

65625 

3644300 


4 

107313 

5365650 

110001 

3661933 

2688 

67200 

3729133 


5 

109750 

5487500 

112500 

3745125 

2750 

68750 

3813875 
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Table 4 

Straight Net Income 


Des. 


Cash 

Line 

Net Income 

After Tax 

Net Cash 

Di scolint 


Alt. 

Yr. 

Out Flow 

Dep. 

Before Tax 

0 48% 

Flow 

0 8.75% 

NPV 

1 

0 





^00000 

1.0000 

-$200000 


1 

$3693000 

$40000 

$1267000 

$658840 

698840 

.9195 

642583 


2 

4110700 

40000 

1399900 

727948 

767948 

.8456 

649377 


3 

4528400 

40000 

1532800 

797056 

837056 

.7775 

650811 


4 

4946100 

40000 

1665700 

866164 

906164 

.7150 

647907 


5 

5363800 

40000 

1834600 

953992 

993992 

.6574 

653450 







Total NPV = 

3,044,128 

2 

0 





-$200000 

1.0000 

-$200000 


1 

3339326 

40000 

1620674 

842750 

882750 

.9195 

811689 


2 

3525852 

40000 

1711748 

890109 

930109 

.8456 

786500 


3 

3709621 

40000 

1805579 

938901 

978901 

.7775 

761096 


4 

3898904 

40000 

1893896 

984826 

1024826 

.7150 

732751 


5 

4085430 

40000 

1984970 

1032184 

1072184 

.6574 

704854 







Total NPV = 

3,596,890 

3 

0 





-$200000 

1.0000 

-$200000 


1 

3405182 

40000 

1554818 

808505 

848505 

.9195 

780201 


2 

3529964 

40000 

1612836 

838675 

878675 

.8456 

743007 


3 

3654771 

40000 

1670779 

868805 

908805 

.7775 

706596 


4 

3779553 

40000 

1728797 

898974 

938974 

.7150 

671367 


5 

3904360 

40000 

1786740 

929105 

969105 

.6574 

637089 







Total NPV = 

3,338,260 

4 

0 





-$200000 

1.0000 

-$200000 


1 

3414084 

40000 

1545916 

803876 

843876 

.9195 

775944 


2 

3564468 

40000 

1614932 

839765 

879764 

.8456 

743929 


3 

3714827 

40000 

1684023 

875692 

915692 

.7775 

711950 


4 

3865211 

40000 

1753039 

911580 

951580 

.7150 

680380 


5 

4015570 

40000 

1822130 

947508 

987508 

.6574 

649187 







Total NPV = 

3,361,391 

5 

0 





-$200000 

1.0000 

-$200000 


1 

3770875 

40000 

1189125 

618345 

658345 

.9195 

605348 


2 

4131856 

40000 

1298043 

674982 

714982 

.8456 

604589 


3 

4492750 

40000 

1407000 

731640 

771640 

.7775 

599950 


4 

4854049 

40000 

1515601 

788112 

828112 

.7150 

592100 


5 

5214625 

40000 

1624875 

844935 

884935 

.6574 

581756 







Total NPV = 

2,783,744 

6 

0 





-$200000 

1.0000 

-$200000 


1 

3855795 

40000 

1104205 

574187 

614187 

.9195 

564745 


2 

4185715 

40000 

1196035 

621938 

661938 

.8456 

559735 


3 

4515635 

40000 

1287865 

669690 

709690 

.7775 

551784 


4 

4845555 

40000 

1379695 

717441 

757441 

.7150 

541571 


5 

5175475 

40000 

1471525 

765193 

805193 

.6574 

529334 







Total NPV = 

2,547,169 

7 

0 





-$200000 

1.0000 

-$200000 


1 

3771680 

40000 

1188320 

617926 

657926 

.9195 

604963 


2 

4154710 

40000 

1302590 

677347 

717347 

.8456 

606588 


3 

4537765 

40000 

1416785 

736728 

776728 

.7775 

603906 


4 

4920795 

40000 

1531055 

796149 

836149 

.7150 

597846 


5 

5303850 

40000 

1645250 

855530 

895530 

.6574 

588721 







Total NPV = 

2,802,026 

8 

0 





-$200000 

1.0000 

-$200000 


1 

3474725 

40000 

1485275 

772343 

812343 

.9195 

746949 


2 

3559525 

40000 

1522325 

791609 

831609 

.8456 

703209 


3 

3644300 

40000 

1559450 

810914 

850914 

.7775 

661586 


4 

3729133 

40000 

1596516 

830189 

870189 

.7150 

622185 


5 

3813875 

40000 

1633625 

849485 

889485 

.6574 

584747 


Total NPV = 3,118,676 


LCS 350:30:000 
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INTRODUCTION 

During the past twenty years, there have grown up a number of procedures to 
identify and solve problems. The ones this paper specifically examines are Kepner- 
Tregoe Analysis (KT), Quality Control Circles (QC@), and Zero Defects (ZD). This is 
not to say that there are not other systems. We have chosen these three chiefly 
because they are fairly well known and some of you may work in firms which presently 
use them. The paper looks at these techniques in light of how each one works to 
solve problems. That is, how does each define the term problem; what kinds of prob- 
lems does each attack; who solves problems; what procedures and tools does each 
technique use; what type of training, if any, do the problem-solvers receive; and 
what is the creativity-quotient of each method. It will be helpful, before we go on 
into the body of the paper, if first we define ’problem 1 and ’problem-solving’. Then 
it will be easier to see how well the different techniques fit the definitions. We 
can view the term ’problem’ in the following ways. A problem is a discrepancy or 
difference between an actual state of affairs and a desired or ideal state of af- 
( 1 ) 

fairs. Another way of saying it is, that a problem is a ’’gap between current 

( 2 ) 

results (or ends) and desired results (or ends)”. A problem exists whenever there 
is sin imbalance between what should be and what actually is; or whenever there is a 
( 3 ) 

deviation from a standard. The key words in these definitions are ’deviation’, 
'gap', 'imbalance*, 'discrepancy', 'difference*. Quite simply, problem-solving is 
the means we use to reduce the gap, imbalance, or difference between what is and 

(4) 

should be. It is how we get to where we want to go from where we are. 

DEFINITION OF PROBLEM 

KT : The definition of ’problem’ contains two corollary ideas. The first says 
that a problem is a deviation from a standard. On the other side is the idea of 

(5) 

problem being the imbalance between what actually happens and what should happen. 

QC® : The concept of problem contains the idea of deviation from a standard as 
well as that of gap between the actual and ideal. This view of what constitutes a 
problem appears to be the same as in KT, but there is a subtle difference in the two 
which shows up in the kinds of problems each procedure addresses. 

ZD: Problem is seen in a specific, all-be-lt limited, way, i.e. as deviation 

(6) 

from a set standard, perfection, and such deviation results in errors or defects. 

In other words, if every employee did his/her job right there would be no errors, no 

(7) 

defects. 


KINDS OF PROBLEMS ATTACKED 

KT : The type of problem KT addresses is by and large the sporadic one. This 
is evident in the KT definition of problem since this definition includes the con- 
cept that the deviation or imbalance is always caused by an unexpected, unplanned 

( 8 ) 

change. This view implies that something has occurred at a specific point in time. 
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Up until the change, the process is acceptable. KT procedure is an example of end- 
product-well-defined because its aim is to stop the sporadic happening and to return 
to the predetermined standard. KT works well in the area of management-controllable 
problems such as in the example of the blackened filaments, chapter two of the 
Rational Manager . 

QC® : The basic thrust of activity aims at the solution of relatively low level 
problems which are chronic in nature. That is the discrepancy between actual and 
ideal or the deviation from standard has been happening over a period of time and 
may or may not have been occasioned by a marked change. The process may be operat- 
ing up to standard but may not be at ideal performance. Moreover the- process itself 
may cause the discrepancies between actual and desired and/or deviations from stand- 
ard. These conceptions differ from the KT approach in-as-much as KT addresses a 
different sort of problem. 

( 9 ) 

Generally the problems handled by the Circles are operator-controllable. 

The operator knows what he/she is to do and what he/she actually is doing and has 
the means to regulate the job. Consequently the Circle members themselves find, 
analyze and implement solutions to work-problems such as defects or scrap reduction. 
QC Circle activity looks to the long-run performance. It solves the problem and 
initiates follow-up, guidelines and safeguards for the solutions. 

( 10 ) 

ZD: This program works to reduce defects by prevention. ZD theory says that 
defects are operator-controllable. Since the operator already has all he/she needs 
to do the job, knowledge and tools, there will be no errors, i,.e. defects, if only 
the employee cares enough to work accurately according to pre-established stand- 

( 11 ) 

ards. 


However, there are certain kinds of problems workers under a ZD program cannot 

( 12 ) 

address. These are ones which arise from difficulties in the process itself. 

These types of problems are routinely handled by QC Circle whereas, in the ZD pro- 
gram, they are handled by supervisors, ZD administrators, or production or quality 

( 13 ) 

engineers. 


WHO SOLVES PROBLEMS 

KT : Essentially it is the manager who is the problem-solver. He is responsi- 
ble for analyzing the problem, making the decision to put to work whichever solution 

( 14 ) 

he has selected, and preparing preventive action. From others who are involved 
with the area under study, the manager elicits help and information in analyzing and 

( 15 ) 

solving the problem. 

QCQ : Problem solving in the Circle is primarily a team effort. The Circle 
is a small group of people who work in the same area, together with their foreman/ 
supervisor. They usually choose the particular problem themselves, then analyze it, 
find and implement solutions. Participation by each member of the Circle is essential 
and everyone’s responsibility. The Circle functions as a team and "stars" are not 
encouraged. There may be interaction with Circles in other work areas if the prob- 
lem is that broad. For example, a Circle in the warehouse might work with a Circle 
in final assembly to prevent cracking of glass tanks. Personnel from quality assur- 
ance, engineering, or other areas serve as resource people. 

ZD: This is a broadly based program which applies to all levels of the organ- 

~ ( 16 ) 

ization. In one sense each person in the organization is a problem solver because 
he strives to improve his own performance, to work without error. However, in the 
sense that problem solving is used in this paper, the responsibliity for solving 

( 17 ) 

problems rests with the supervisor and/or ZD Administrator and a team which is 
not directly involved with the work area. The supervisor corrects problems discov- 
ered by the workers and reported on the Error Cause Identification (ECI) form. 

Should the problem prove too difficult or require handling by superiors, the super- 
visor passes to the ZD Administrator or other appropriate officials. The man on the 
line has no responsibility to solve problems other than to remove defects. Should 
he spot a problem in the process, he is encouraged to fill out an ECI form and for- 


ward it to his supervisor. If he has thought of a solution, he submits this through 

(18) 

the suggestion system. 

PROBLEM SOLVING SEQUENCE 

(19) 

"Problem solving is a process that follows a logical sequence." The process 
by whatever name contains familiar and recognizable steps: once the problem is 
recognized, the problem-solver looks for causes. He collects information about it 
from what he observes. Next he speculates about the causes, analyzes them, develops 

( 20 ) 

solutions, chooses one and puts it to work, always revising as necessary. 

KT : Messrs. Kepner and Tregoe have developed a very orderly and precise method 
for analysis of problems, for making decisions and obviating potential difficulties 
in the solution. We give here a brief summary of this procedure, based on material 

( 21 ) 

from Results Planning Manual , the KT workbook. The procedure has three parts, 
Problem Analysis, Decision Analysis, and Analysis of Potential Problems. Problem 
Analysis is the systematic way of finding the real cause of the problem. First 
specify exactly what the problem is and is not, noting areas of sharp contrast. 

Next develop possible causes . Test each possibility to determine the probable 
cause. Verify the results. 

Decision Analysis is the method used to choose the best solution to the prob- 
lem. First, establish the objectives , what are the desired results and what are the 
necessary resources. After the objectives are clear, classify as must : the critical 
area; and want : those things of relative importance. Next, find alternatives . 
Compare these to the objectives for consequences. 

Potential Problem Analysis anticipates potential problems and devises plans to 
prevent their occurrence or to protect the solution from the impact of the potential 
problem if it should occur. 

QC0: The first step in the problem solving sequence followed by the Circles is 

( 22 ) 

to identify goals . Usually the goals are fairly broad, such as reduction of scrap 
or defects in a particular department. In order to narrow the goal to a workable 
size, the Circle breaks down the general problem area into its components. At this 
stage the Circle collects data by means of check sheets, tallies, etc. The Circle 
proceeds to analyze the data by means of Pareto analysis which sets priorities and by 
frequency histograms. In determining causes, the Circle employs brainstorming , and 
records ideas on a Cause and Effect diagram (C 8c E). Incidentally, it is well to 
note Kepner' s and Tregoe' s caution about brainstorming that the technique "does not 
lead to an understanding of precisely what is wrong, how things got that way, and 

(23) 

what is the most economical way of correcting the trouble." When the Circles 
brainstorm, they are looking at four precise areas where trouble can arise: men, 
method, material or machine. By using the C & E Diagram, the Circle members can 
graphically see the interrelationships of different aspects of the problem under 
analysis. 

The next step is to experiment to see which ideas recorded on the C & E Diagram 
are the most likely causes. The circle analyzes these results using graphs and 
other statistical analysis. Once a solution is selected, the Circle tests and imple- 
ments it. The Circle also designs and implements safeguards to make sure the solu- 
tion continues to function properly. 

In some respects this problem-solving sequence resembles the KT approach. The 
Circles specify the problem through statistical tools such as Pareto analysis, fre- 
quency histograms or check sheets. It is in the brainstorming phase that the Cir- 
cles develop possible causes. Both KT and the Circles test for the probable cause 
and verify the results. 

The KT procedure seems to follow a more definite pattern in order to determine 
what the solution is than do the QC Circles. The Circles have selected a goal 
beforehand, such as reducing per cent defective parts, before analyzing the causes. 
Both procedures choose solutions. The Circles test the solutions to find potential 
problems. By designing and implementing safeguards, the Circles, just as KT analy- 
sis, provide for protection and contingent action. 



The dissimilarities in the two approaches appear because the methods deal with 
different orders of problem. The QC Circle solves low level, chronic problems; KT 
handles sporadic serious problems. 

(24) (25) 

ZD: The ZD program seeks to motivate or encourage people to do their 

( 26 ) 

job right the first time, every time. Generally speaking ZD aims toward the solu- 
tion of a single specific problem: reduction of defects by prevention rather than 
(27) 

through detection. The program was not designed to solve problems nor was it in- 
tended to teach techniques for solving problems. 

Even so a type of problem solving procedure does take place in Error Cause 
Identification. The employee recognizes that a problem exists which either does, or 
has the potential to, cause errors. He fills out an ECI form which he submits to his 
supervisor. The main responsibility for solving the problem rests with the super- 
(28) 

visor. Then the supervisor verifies that a problem exists, determines what must 

be done to correct it, and then puts into operation whatever solution he has chosen. 
In some instances the quality control representative and possibly an assigned team 
(29) 

find the solution. 

We would like to take a look at an instance of problem solving in the example 

(30) 

given by J. M. Halpin in Zero Defects . The supervisor discovered an increase of 

surface scratches on a product. The cause is supposed to result from careless and/ 
or improper handling. The supervisor speaks to his workers about the scratches. 

When the scratches continue to occur, the supervisor calls in the ZD Administrator. 
This person checks the end result of scratches on the finished product. He finds 
that this defect can cause the ruin of the finished product. The Administrator 
demonstrates to the workers what he has found. 

The manner in which the supervisor handled this particular case, as it is re- 

(31) (32) 

ported, points to a type of difficulty noted by Dr. Ishikawa, and Dr. Juran. 

The supervisor assumes that the cause of the scratches is careless handling, but it 
might well have come from defective tools or the process itself. No one verified 
the cause using standard statistical methods. 


TRAINING IN PROBLEM SOLVING 


KT : The manager receives training in KT analysis through study of the prin- 
ciples as outlined in The Rational Manager and through participation in a KT train- 
ing session. An essential part of this training is the practice of the concepts in 
simulated and personal business situations. These practice sessions are critiqued 
( 33 ) 

by the KT instructor. 

QC0 : The supervisor receives extensive training in group dynamics and basic 
statistical tools: histograms, Pareto analysis, control charts, stratification, 
binomial probability paper, cause and effect diagrams, graphs, etc. He, in turn, 
trains the people who work under him in these same techniques. Training is not 
limited to job training, honing of work skills or improving understanding of the 
process. Workers may indeed receive this sort of training if the Circle finds 
that more job training is part of the solution to the problem it has tackled. 

The skills that the worker in a QC Circle learns are practical statistical con- 
cepts. In KT the manager learns an efficient method for breaking down a problem 
and deciding how to solve it. He would further enhance his problem solving skills 
if he included some statistical analysis. KT teaches a problem-solving methodology 
which enables an individual to handle problems successfully. By contrast the sta- 
tistical skills allow the Circle to work as a team with great effectiveness. 

ZD: There is minimal training for problem solving for supervisors and workers, 
aside from instruction in the proper use of the ECI form. The expectation is that 
either the supervisor will correct the problem identified on an ECI form or someone 

( 3*0 


else will 


CREATIVITY QUOTIENT 


Writers, as widely different as J. R. R. Tolkien, Francis Schaeffer, J. M. Juran, 
George M. Prince and Prof. Y. Kondo recognize that creativity is a basic human 
(35) 

right. Programs and procedures which challenge people to do their job creatively 

and give them the methods to do it will be the most successful and will do much to 
improve the quality of work life. Creativity cannot perhaps be quantified, yet such 
things as increased employee motivation and reduction of boredom and absenteeism are 
the kinds of effects a program of problem solving is likely to have. People need to 
perceive they have some control over their work even if such control is limited,. 

We are outsiders: we doh’t manage a factory nor do we work on line. But as 
outsiders, we have a perspective which can focus on strengths and weaknesses of the 
various procedures. A program which helps to sharpen a worker’s skills and teaches 
him how to use them will have a high creativity quotient. When a person is better 
prepared to handle what comes to him in life, whether at work or at leisure, he 
develops more self confidence that he can participate in the control of his circum- 
stances. KT and the QC Circle concept do these things. They recognize that people 
are problem solvers and that they need training to be effective. KT gives a logical, 
efficient method of breaking down a problem and arriving at a solution yet the pro- 
cedure is flexible. KT is generally used on big, one-shot problems, but the techni- 
que can be applied in almost any situation. One strength this procedure has is that 
the problem-solver goes to the problem and lets It speak. Usually it is a single 
individual who uses KT analysis rather than a team; but there is no reason why a 
team cannot use this method to solve problems. 

KT is designed for managers, though if a manager is people-oriented he will be 
concerned with those under him, that they also become strong problem-solvers. Con- 

(36) 

sequently he will teach the KT analysis principles to his subordinates. 

The QC© concept emphasizes the statistical analysis of problem solving. Because 
it is a team effort, each member has an opportunity to contribute, to have his ideas 
accepted, to be creative, and so to develop self worth. Through the statistical 
analysis the team participates in the control over the work process; they are able 
M to close the loop”, they can change what they are doing to what they are supposed 
(37) 

to be doing. 

One of the strengths of the ZD program is that it covers the entire organization 
from management on down to the man on the line to make each person aware of his 
responsibility to do his job right. KT trains managers; the QC Circle trains super- 
visors and workers. 

A caution we would post for the ZD program is one already anticipated by 
Dr. Juran. As long as the workers already have in their hands the proper training 
to do the job, knowledge of the process, and the means to change their actions or 
the process in order to conform to what is expected, a ZD type program can help 
(38) 

motivate workers. 
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"ZERO-DEFECTS” - ALIVE AND WELL IN THE EASTERN EUROPEAN COUNTRIES 
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INTRODUCTION 

Back in the early 1960's a motivational program to raise product, process, 
and service quality levels, called zero-defects (ZD), swept the country. Unfortu- 
nately, in the author's opinion, it was mis-applied and, as fast as it swept up 
to great heights of recognition and application, it fell even faster into disrepute 
and discard as another "fad". However, the application of the ZD concept was 
not unique to the U.S.A. The ZD concept has been "alive and well" for some years 
in the Eastern European Socialist countries, and is currently enjoying a resurgence 
of recognition and application. 

The author was informed - but was unable to verify the fact - that the ZD 
concept was applied at the Bata Shoe Company in Czechoslovakia prior to World War II 
However, there is documentation to support the statement that a ZD concept was part 
of the Soviet Saratov System of Quality Control (1955), and part of the recent 
variant of the Saratov System, called the Lvov System of Quality Control (1967). 

There has been a recent upsurge in interest in these two systems, and 
especially the ZD concept, as a result of the Communist Party Congress. At the 
February - March 1976 Congress of the Communist Party of the U.S.S.R., Mr. Breznev 
enunciated the "line" for the current five-year plan as "Raising Efficiency and 
Quality". The Parties of the other Eastern European Socialist countries adopted 
similar goals for their five or ten-year plans. The result has been a renewed 
interest in the Saratov and Lvov Systems and their implanted ZD concept. 

THE SARATOV AND LVOV SYSTEMS 

Space does not permit a full development of these two systems, but enough 
will be presented to give the reader an idea of the systems, and how the ZD 
concept inter-relates with them. 

Of the two, the Saratov System is the older, having begun around 1955, as a 
movement to motivate workers toward better quality work, in the. machine tool and 
engineering works in the Saratov region of the Ukraine (the System is named for 
the region where it originated). Its central idea was that all human activity, 
from design to selling and service, is the result of human work, and that the 
quality of the activity (resultant product or service) depends upon the quality 
of this work. In addition, it incorporated the idea that every worker is responsibl 
for his/her quality of work, and the concept of self-regulation of quality (be the 
inspector for one's own work). 

The System divided the product cycle into three stages - preproduction, 
production, and post-production. Then, for each stage and for the various type 
of workers at each stage, e.g., designers, technologists, production workers, 
inspectors, a criteria for defect-free work was developed. This criteria was the 
basis for the quality level, indicator, which was essentially a measure of the 
percentage of defective work. These indicators were applied to various levels, e.g. 
individual, professional and trade groupings, stages (preproduction, production and 
post-production), factories, enterprises, industries and ministries. 


*Professor Mihalasky spent the 1976-77 academic year as a Scientific Exchangee 
for the U.S. National Academy of Sciences, attached to the Institute of Information 
Theory and Automation, of the Czechoslovak Academy of Science, Prague, 
Czechoslovakia 
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These indicators could also be based upon more complex measures than just the 
percentage of defective work. For example, the following eight factors made up 
one such quality level indicator: 

1. Percentage of good product upon first submission. 

2. Fulfillment of the monthly plan for the improvement of product quality. 

3. Breech of technological methodology. 

4. Loss from spoilage and scrap. 

5. Product rejected by plant inspection. 

6. Product rejected by inspections further down the stream of commerce. 

7. Customer complaints. 

8. Adherence to the "culture of work" (workplace conditions, general 
quality level of work produced, safety, morale, etc.). 

The Saratov System became quite popular and soon spread to the rest of the 
Soviet Union, and to the other member nations of the Socialist Bloc. The Saratov 
System became known by the initials SBIP or BIP which stood for "Defect-Free 
Finished Product". 

However, as the movement spread, variations began to appear. There appeared 
the Jaroslav, Minsk and Lvov variants in the Soviet Union, and other variations 
such as the Mlada Boleslav movement in Czechoslovakia, when a "home-grown" 
variety was desireable. These variations were in addition to changes made to 
the classical Saratov format. Of the Soviet variations, the Lvov System known 
by the initials SBT meaning "Defect-Free Work", and developed at a telegraph 
apparatus plant in the Lvov region of the Ukraine in 1967, is the most well known 
in Eastern Europe. 

The reason(s) for the Lvov and other variations are varied, and to some extent 
unclear and confusing, because in some cases the updated Saratov System installa- 
tions are difficult to tell apart. Some of the reasons for the development of the 
Lvov System, and its difference from the Saratov System were: 

A. The Saratov System was aimed more at manual workers - it was not 
broad enough in its coverage of non-factory workers. 

B. The Saratov emphasized "Defect-Free Finished Product", while the Lvov 
System emphasized "Defect-Free Work". 

C. The Lvov System developed a more complex, quantitative set of indicators 
for the quality levels of workers, Brigades, collectives, industries and 
ministries . 

Both the Saratov, Lvov, Mlada Boleslav and other variant Systems are designed 
to be total Quality Control systems which have the ZD concept as one of their 
integral parts. 


THE ZD CONCEPTS 

The following are some of the characteristics of the ZD concept which these 
Eastern European Quality Control Systems have in common. 

First of all, the system, especially the idea of "Defect-Free Work" and 
"Defect-Free Finished Product", was to be backed by the mobilization of all "forces" - 
the Party, the Labor Unions, the Ministerial Offices, the Management, the Collectives 
and the Brigades. The direction, including the determination of quality .level 
standards and indicators to be maintained, emanated from the ministerial level. 

Both moral and material stimulii were to be used. Moral stimulii consisted 
of such as patriotism - for good of Party and Country - and Socialist Competition - 
essentially striving for a first level rank among brigades, collectives, factories 
and/or industries. Material stimulii would take the form of positive and negative 
supplements to premiums or bonuses. 

"Quality Days" were to be periodically held - once a week to once a month - 
when quality level indicators would be reviewed, and reasons for poor quality 
developed and analyzed for later correction. 


An especially important aspect of the ZD approach was the potential that the 
worker had of attaining the level where he inspected and passed on his own work. 

Under the early Saratov System a worker that submitted defect-free work for a six- 
month period could attain the rank and title of "Self-Inspector" or "Self-Controller". 
This gave him the right to inspect his own work and pass it along without having to 
wait for the usual quality control inspector and inspection. It also provided him 
with a bonus of 10% of his pay. However, the "self-inspector" was subject to an 
audit about twice a month, and if found producing defective work, was subject to the 
loss of his rank and privileges, along with the bonus. 

Under the Lvov and the more modern variants, the "self-inspector" concept was 
modified so that more people could participate. In this more modern version, a 
selected worker could voluntarily join the "Guild for Exemplary Quality" and assume 
the role of a self-inspector with a title such as "Producer of Exemplary Quality- 
Self Inspector"-. In return, the worker agreed: 

1. To work in the spirit of good quality production. 

2. Received a personal stamp with which all of his work would be identified. 

3. Had the right to participate in meetings and discussions on scrap 
and defect control (as long as his own work wasn't being discussed). 

A. Had the right to quick and complete information about the quality 
level of his work - especially rejected work. 

5. The right to use the efforts and information of the quality control 
technologists . 

6. Would receive a monthly bonus-varying from a percentage of base salary 
to a flat monthly amount. 

This position was maintained as long as the worker continued to produce good 
quality work, as measured by the system's quality indicators. 

The "self-inspector" would perform such inspections as he was capable of 
without special training or equipment - visual inspection, use of basic measuring 
tools, non-destructive tests. The work would then be passed down the line to 
where the technical control group, e.g., Quality Control, would perform the more 
rigorous inspections. 

To aid in maintaining quality and assigning the responsibility for defects, 
a defect classification system, a defect list that also contained the party or 
parties responsible for the defect, and a daily posting of defects uncovered by 
inspection were used. 

All new workers were indoctrinated in the concept of "defect-free work", and 
impressed with the thought of "Do it right the first time". 

CONCLUSION 

The application of the Saratov, Lvov, and other variant systems ranged from 
machine tool manufacturing industries to the food industries to research institutes. 
The concept of worker responsibility was pressed especially hard, and is reminiscent 
of the experience with ZD in the U.S.A. where "the tail wagged the dog" - ZD was 
thought to be superior to, and separate from, a good total quality control system, 
rather than as a tool to be used within the scope of a good quality control system. 

With the current emphasis on Efficiency and Quality in the current five and 
ten year plans, the application of the ZD concept has not as yet reached its peak, 
so the time to evaluate the success of the movement has not yet arrived. 
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THE PROBLEM WITH QUALITY 


Ervin F. Taylor 

Corporate Director Quality Control 
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INTRODUCTION 

To deny that a problem with quality exists would force one to admit 
to reading no newspaper, watching no television, listening to no radio — 
or — purchasing no products or services. The famous industrial designer, 
Raymond Lowey wrote: 

Americans are in a massive race to maintain our leadership 
against other nations in trade, training, goods and taste. 

This is a fight. . .measured more in actions than in words, 
and one where our country loses a little each time any one 
of us lets his standards fall. 

This fight is lost in 'little' ways — by faucets that leak. . . 

windows that rattle. . .windshield wipers that don't wipe. . . 
lights that flicker. . .zippers that jam. . .engines that 
smoke. . .containers that leak. . .flags that fade. . .by 
salesmen who are rude. . .by repairmen who are untrust- 
worthy. . .artists and artisans who do their second best. . . 

by designers and manufacturers who think it doesn't matter 
just this once if they turn out products which are vulgar, 
shoddy and overpriced. ( 1) 

When was this written? Yesterday? No. It appeared in June 1962. 

It sounds so familiar because the situation has changed very little. 
Consumers continue to have a problem with quality. 

CONSUMER PROBLEM WITH QUALITY 

The fact that every individual is a consumer who purchases products 
and services means that many millions of times each day there is an op- 
portunity for a problem with quality. The exposure rate is quite high. 

Think back to the last time you had a complaint about the quality of 
a product or service. What was the complaint? Was someone injured? 

Was there a failure to function the first time tried? Was the time- 
to-failure too short? Was the construction poor? Was the design 
inconvenient for use? Was the finish shoddy? Was the packaging 
inadequate? Were you treated discourteously? 

Each of these possible complaints has a common element. Each 
represents a failure to meet your expectations. You had expected some- 
thing; but were given less. At this point it is not important whether 
or not your expectations were valid. The fact is that these expectations 
do exist. This leads us to Business Quality Law No. 1. 

Business Quality Law No. 1: If the customer believes that a product 

is of poor quality, the product is in 
fact of poor quality. 

If a customer believes that a product is of poor quality, that 
consumer probably will not buy that product again. Furthermore, the 
customer may also cause the loss of additional sales through influence 
on relatives, friends and acquaintances. Thus, the poor quality results 
in a business loss even though the product met all of its drawing and 
specification requirements as evidenced by a thorough inspection. 
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Consumer expectations are higher today than they have been previ- 
ously. Two major factors have contributed to these higher expectations. 
First, the massive aerospace programs with their rapidly expanding tech- 
nology have created in the consumer an impression that "all things are 
possible". As new breakthroughs are announced, the consumer greets them 
with a blase attitude of no surprise — it was expected. This posture has 
been transferred to even the ordinary products with which the consumer 
comes into contact. 

Second, the "consumer movement" which was given considerable impetus 
in 1962 by President Kennedy's proclamation of the "Consumers' Bill of 
Rights", has grown into a major business. Where only a few years ago 
there was only the Better Business Bureau, there are now more than 300 
Government consumer offices at state, county and city levels. Add to 
this the 31 federal agencies with consumer offices, 38 industry and 
private recourse groups, 6 national consumer organizations and 50 "Call 
for Action" programs at radio and television stations nationwide.^) 

This does not include the innumerable businesses who have added consumer 
relations organizations. 

All of this means that today's consumer is increasingly bombarded 
with information and solutions on all manner of problems. The result 
tends to inculcate in the consumer the impression that whatever he wants, 
he can get. 

The consumer's problem with quality is simply that products and 
services do not measure up to his expectations. 

THE BUSINESS MANAGER'S PROBLEM WITH QUALITY 

Like the consumer, the business manager's problem with quality is 
failure to meet expectations. However, unlike the consumer, the business 
manager's problem is a complex, multi-faceted problem requiring precisely 
balanced solutions. 

The business manager's problem with quality can be described as a 
problem with Government regulation, product liability, quality sales 
losses, quality costs, or any combination of these factors. 

Government regulations at the federal, state and local levels are 
having an increasingly significant impact on virtually every business. 
Requirements are placed not only upon products and services, but also 
upon the business systems which assure that the requirements are achieved. 

Companies with contractual business with the Federal Government 
have been accustomed to both product and business system requirements. 
Product requirements are defined either in standards, specifications or 
in the contract. Business system requirements — especially those related 
to controls for product quality — are delineated in specifications such 
as Department of Defense MIL-Q-9858A, "Quality Control System Require- 
ments"^) and National Aeronautics and Space Administration NHB 5300.4 
(IB), "Quality Program Provisions for Aeronautical and Space System 
Contractors ." (4) Since these requirements are contractually imposed, 
the cost impact on the business is a determinable factor which can be 
negotiated. 

A much different problem is faced by those businesses not producing 
products to contract, and are regulated by an agency such as the Food 
and Drug Administration (FDA), or the Consumer Product Safety Commission 
(CPSC). Both of these agencies regulate through Congressional mandates 
citing certain "prohibited acts" which are violations of the law. In 
the case of the CPSC, penalties are either civil or criminal. For 
businesses regulated by the FDA, penalties for violation of prohibited 
acts are criminal violations. 

Both the FDA and CPSC have the authority to regulate product quality 
through product performance standards. The Consumer Product Safety 
Commission regulates the business system for quality control through 
"voluntary implementation" of a "Handbook and Standard for Manufacturing 
Safer Consumer Products ."( 5 ) The objective of these guidelines is to 


to assist the manufacturer in developing business systems which will 
prevent the occurrence of prohibited acts. 

The Food and Drug Administration takes a different approach by pro- 
mulgating "Good Manufacturing Practices" (GMP) regulations for a class 
of products (foods, drugs, cosmetics, medical devices) and/or for 
specific products (candy, meat, prescription drugs). Violations of 
GMP's render a product "misbranded", and misbranding is a "prohibited 
act" subject to criminal penalties. 

The manager of FDA and CPSC regulated businesses must consider the 
cost of compliance with the regulations in product pricing. The problem 
is not, however, straightforward since compliance would presumably reduce 
other costs such as product liability. 

One of the most severe problems with quality facing the business 
manager is product liability. Since the doctrine of strict liability 
for manufacturers was upheld by California Courts in 1963, both the 
number of product liability suits and the amount of the awards has 
escalated. In 1965 the average award was $11,644. By 1973 the average 
award had increased to $79,940. In 1975 in Cook County, Illinois, the 
average award was $144,091 and this excluded one judgement for 
$5,000,000! In virtually every case the products involved were cited 
directly or indirectly for poor quality which caused or contributed to a 
hazard. 

According to a 1976 report (?), there were 20,000,000 injuries in 
1973 which required medical treatment. Of these 110,000 resulted in 
permanent disabilities and 30,000 in deaths. Involved were some of the 
11,000 consumer products and 360,000 manufacturing establishments. The 
estimated loss to the U.S. economy was $5.5 billion. 

Statistics such as these are startling, and would appear to add 
significantly to the problem with quality. However, caution is urged. 
When one considers the exposure rate — the number of people who use con- 
sumer products, times the number of products they use per day, times 
the number of uses per day, times the number of days per year — a 
different picture emerges. Assume that the above injuries were sus- 
tained by 190,000,000 Americans using each of 10 products twice each 
day. This exposure rate results in an injury rate of slightly more 
than 0.001%! The business manager's problem with quality as it con- 
cerns product liability is how can cost effective controls be exercised 
which are so precise as to reduce an already low injury rate? 

The business manager's problem with quality does not stop with the 
regulatory and liability impacts. The products or services being pro- 
duced are compared in the marketplace with similar products and services. 
One of the major areas of comparison is how well does this product meet 
the expectations — the product quality — relative to similar products. If 
the product suffers in the comparison, sales will suffer. The business 
manager must somehow uphold product quality while at the same time hold- 
ing down prices to remain competitive. The balance of these two factors 
can be extremely critical. 

There is still another problem with quality which plagues the 
business manager. This is the constant pressure by managers and engi- 
neers in the professional field of controlling quality to add more 
inspectors, to buy more test equipment and to establish more and more 
sophisticated techniques on more and more areas of the business. The 
business manager is told that these things are essential to produce 
products with fewer problems with quality. 

This escalation of the "quality control empire" is quite visible 
in many companies; and it is fostered by the regulators, by the liabi- 
lity threats and by the competition. The "value" of a program for 
quality control is too frequently cited as being self-evident. Rarely 
are the costs and risks estimated so that a meaningful management 
judgement can be made. 


The business manager's problem with quality is obviously extremely 
complex. Unless the problem is solved--the product, the service or 
even the business may not survive. 

THE QUALITY CONTROL PROFESSIONAL'S PROBLEM WITH QUALITY 

The quality control professional is one who has developed an exper- 
tise in the techniques of measuring and controlling quality. It might be 
assumed that this class of individuals would have few if any problems 
with quality. Such is not the case. 

One of the first problems with quality encountered by the quality 
professional is a conflict with the business manager. A measurement of 
quality indicates that a product fails to meet the requirements. The 
product is rejected. The business manager orders the product shipped, 
thus "over-ruling" the decision prompted by the measurement of quality. 
This can be a severe problem with quality, and has resulted in more 
than a few resignations of quality control professionals. 

The business manager is responsible for the profit and loss of the 
business. Decisions regarding quality should be predicated upon not 
only the measure of quality; but also the business risks of shipping 
or not shipping a product which does not measure up. The answers to 
the two risk questions: "What is the business risk if the product is 
shipped?" should be based upon more than an intuitive judgement. Un- 
fortunately, this is not always the case. The business manager may 
have a sound assessment of the risks of nonshipment ; however, a know- 
ledge of the risks of shipment, which are -more technical in nature, 
may be meager. The quality professional who does not provide an 
estimate of the risk as well as the measure of quality, is contributing 
to the problem with quality. 

Another of the quality control professional's problem with quality 
is the lack of management support for a program to control quality. 
Papers on this problem frequently appear in the literature and are pre- 
sented at symposia. Corridor conversations at such meetings very 
frequently involve this problem. The typical cry is that management 
looks on engineering and manufacturing as contributing value to a 
product; but they look upon inspection and quality control as not con- 
tributing value to the product, as being obstructionists, or at best, 
as being a necessary evil. Management tends not to think of quality 
control as a part of the business. And the primary reason for this 
attitude is that many, if not most quality control professionals 
rarely think of their function in this context. 

There are numerous indications of this "quality control is not a 
part of the business" attitude. There is constant pressure from both 
outside the business and within the quality control profession to keep 
the quality control function independent of other business functions 
such as manufacturing, engineering, marketing, etc. There is a tendency 
for quality control professionals to look at each new product regulation 
as a reason to expand the quality control organization. A look at the 
quality control organization in many companies will reveal a quality 
control empire where virtually all of the quality controlling functions 
of the business are snatched into this organization. 

Quality control professionals frequently talk cost savings; how- 
ever, when asked for specifics — when asked if the value of a. quality 
controlling function exceeds the cost — respond with glittering gener- 
alties and "motherhood" statements. 

Quality control professionals do have a problem with quality. 
Unfortunately, too many quality control professionals are a part of 
the problem with quality. 



TOWARD A SOLUTION OF THE PROBLEM WITH QUALITY 


A solution to the problem with quality can only be achieved through 
action--act ion on the part of the consumers, business managers and qual- 
ity control professionals. The following are some of the actions which 
should be taken: 

Consumer Action 


1. Recognize that, as a consumer, your problem with quality is simply 
that expectations have not been met. Determine first if your 
expectations are realistic. You can place your expectations in 
throe categories: 

a. Rea.listic - the actual must be changed. 

b. Unrealistic - the expected as claimed (advertising, salesmen, 
etc.) must be changed. 

c. Unrealistic - the actual meets the expected as claimed; but 
still is too low. 

2. Complain first directly to the business involved. The odds are in 
your favor that you will get satisfaction on the first attempt. 

There are techniques for complaints which will enhance your chances 
of satisfaction. These techniques are very well-described in 
"Complaint Power"( 3 * * * * 8 ), a 24-page publication of the Maryland Center 
for Public Broadcasting Consumer Survival Kit award-winning pro- 
gram now in its fourth season. The recommended steps are: 

a. Be sure the complaint is justified. 

b. Decide what action you want. 

c. Contact the person who has the authority to make the deci- 
sion you want . 

d. Keep the tone business-like. 

Unfortunately, too many consumers complain to everyone excepting the 
business involved. The businessman deserves to hear your complaint 
from you first. In most cases, you will get satisfaction. However, 
if the businessman fails to respond, do not hesitate to utilize the 
many other resources available to you. 

Business Manager Action 

1. Respond to each customer complaint promptly, in a business-like 
manner and with positive action, where feasible. If action cannot 
be taken on the customer’s problem, inform him of the reasons. If 
action must be delayed, inform him of when it will be taken. 
Remember, action can be taken to change either the product or 
service, or the expectations. 

2. Examine your product literature and advertising for statements which 
tend to over-inflate the customer's expectations. Periodically 
audit sales talks for inflated claims. 

3. Consider your quality control function as an integral part of the 

business. Insist that standards realistically match customer ex- 

pectations, and avoid "perfection" as a standard. Require quality 
control managers to show that people, measuring equipment and pro- 
cedures do in fact reduce long term costs. 

Quality Control Professional Action 

1. Think of your function as an integral part of the business. Apprise 

those with profit and loss responsibility of the risks, as well as 
additional personnel, equipment and procedures. 



2. Avoid adopting a new program developed by a source outside your 
business, unless that program can be shown to decrease Ipng term 
costs for your business. Such programs include sampling systems, 
motivation programs and methods to specifically meet regulatory 
or other requirements which may not be appropriate for your busi- 
ness . 

3. Work constantly to achieve consistent agreement of actuals with 
expectations by assuring that realistic standards are set and en- 
forced at all levels of the business process. 

In Conclusion 


If we as consumers, business managers and quality control profes- 
sionals work on both the actuals and the expectations to make them more 
nearly equal, it will not be long before the words of Raymond Lowey 
cited earlier will no longer be true. When that happens, there will 
be no problem with quality. 
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QUALITY, MANAGEMENT AND YOUNG ENGINEER 


H. J. Bajaria, Senior Reliability Engineer 
Rockwell International 
Troy, Michigan 48084 


ABSTRACT 

Most graduating engineers entering the field of quality assurance 
face five basic problems in an industrial environment with respect to 
achieving quality in the final product. These problems are: (1) A 
young engineer is apathetic toward a quantified decision making process, 
(2) He practices the principle of interocular trauma in handling his 
assignments, {3) He lacks an understanding of variation in identical 
units of the same product, (4) He has a built-in zero error attitude in 
his decision making process, (5) He lacks business communication skills. 
This paper discusses (1) the sources of these problems and (2) the 
relation of these problems to the quality of products. 

It is essential for management to recognize the existence of these 
basic problems, to understand them and to apply solution methods which 
will motivate young engineers to overcome them. A motivational approach 
advocated in this paper includes: (1) providing opportunity to young 
engineers for training in applications of statistics to decision making, 
oral communication and written communication (2) communicating to 
engineers that overcoming these problems means success (3) providing 
periodic performance reviews with respect to progress on these problems. 
A case history approach is used to illustrate how to accomplish 
recognition, understanding and solution for each of the five problems. 
Case histories cover consumer industries, government contracts and OEM/ 
supplier situations. 

The paper further discusses the consequences if these problems are 
not recognized or if they are recognized and nothing is done to overcome 
them. In either case, management should be concerned with the cost and 
dangers resulting from inadequate recognition of product quality 
deficiencies. Each area of corporation vulnerability is identified and 
the consequences are discussed. 
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PRODUCT QUALITY IMPROVEMENT THROUGH "VISIBILITY 


Jay W. Leek & 

Fred H. Riley 
Northrop Corporation 
Defense Systems Division 
Rolling Meadows, Illinois 60008 


SUMMARY 

A practical approach for improving product quality is by means of quality visi- 
bility and employee quality awareness programs. Using Northrop Corporation Defense 
Systems Division as an example, this paper expresses the conviction that people 
excel in task performance beyond expectations when given proper management atten- 
tion , a clear understanding of the requirements , adequate resources and timely 
measurement of performance. 

By advocating, implementing, and maintaining an Employee Quality Awareness 
Program and a Quality Visibility System, a significant product quality improvement 
has been reached at the Defense Systems Division (DSD). 

This end has been accomplished through innovative systems that: 

-provide continual quality awareness; 

-develop a spirit of competition among small organizational groups; 

-provide recognition of group endeavor; 

-instill a feeling of personal achievement within the individual; 

-involve all levels of management; 

-provide visibility to all through a centrally located Visibility Center; 

-force timely implementation of corrective action through open display 
of adverse trends as associated with specific responsibilities. 

GENERAL 

The concept of quality visibility (originated at Northrop Aircraft Division) 
and awareness (originated at Northrop Defense Systems Division) was conceived to 
induce a team action in the production of hardware that is superior in quality and 
reliability. 

In the usual concept of quality control, defect and/or test data are gathered 
and circulated by periodic reports which are sent to management personnel of the 
affected areas, or a chart room (war room) is maintained as a display for top manage- 
ment personnel only. 

In either case, the effective response is often negligible. The report Is 
discarded and the war room charts scanned and ignored if they do not contain a 
clearly defined responsibility. 

Under the quality visibility and awareness concept however, quality status, 
cost impact problems, and corrective actions require the responsibility to be de- 
fined. Process area charts (areas of responsibility) are displayed in the Visi- 
bility Center so that they may be recognized as to responsible individual as well as 
to the individual contributor to the quality status of the area. 

Quality awareness is achieved through a continuing effort by management to 
recognize team and individual achievements. 

DSD's Visibility Center and Awareness Program were implemented in 1976, based 
on the premise that when given proper attention, a clear understanding of the re- 
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quirements, adequate resources, timely measurement of performance and recognition of 
contribution, people excel in task performance beyond expectations. 

Since its inception, this program has proven employee team performance in the 
manufacture of a product that is cost effective and superior in quality and relia- 
bility. DSD's record of accomplishment is a direct reflection on the skills of 
individuals who have a personal interest in team performance in the production of 
quality products. 


OBJECTIVES 

Knowing that motivation influences productivity, implementing a program that 
would make the best use of proven techniques was a prerequisite. We invoked a 
number of motivational techniques such as job enrichment, team competition, recog- 
nition of team and individual effort, reward, and participation by all employees in 
the decision making loop. 

To these was added a program or series of programs that provided an awareness 
of quality and the rationale for its need. 

Identification of process area responsibility is maintained in the Visibility 
Center as well as on individual floor charts. Although this method may give an area 
supervisor undesirable exposure, if his area of supervision has an unfavorable 
defect trend, yet it may provide a desirable stimulus by the exposure and natural 
competitive spirit induced between the supervisors of the various process areas. 

In order to meet our objectives, it was necessary to provide an effective 
system that would collect and display manufacturing performance data and provide a 
continuous quick response to effect corrective action when needed and employee 
recognition when warranted. 

To provide the visibility and the management involvement required to assure 
quality awareness, a central area open to all employees was of paramount importance. 
This center would maintain displays of Manufacturing and Procurement quality status, 
cost impact problems, and corrective action follow up as related to a specific 
manufacturing and procurement operations assigned to a Manager or Supervisor re- 
sponsible for their administration. 

In order to maintain quality levels necessary to provide quality goods to our 
customers on a cost effective basis, controls had to be published. 

KEY ELEMENTS 

The following key elements are necessary in order to provide a continuing long- 
term level of quality within the organization: 

A. All levels of management must be involved. Top management must not only 
have a sincere interest in the level of quality, they must also actively 
participate in effecting corrective action at any level. They must also 
be involved in the recognition phase to provide stimulus at the craft 
levels. 

B. Manufacturing and Quality organizations must work together to provide 
continuous process control. 

C. A systematic approach for establishing acceptable quality performance pub- 
lished goals based on process capability. 

D. A centralized data display for the use of management and elucidation of 
the work force. 

E. Timeliness of collection, analysis and utilization of data are imperative 
for an effective corrective action response. 

F. Division of the work force into discrete work unit or process group 
quality achievement teams, and suitable displays of group performance to 
stimulate quality awareness and a competitive spirit. 

G. Recognition must be a continuing effort, equitably based on achievement. 



ADMINISTRATION AND DIRECTION 


The program is best administered by the Quality Engineering organization. 

The program administrator is responsible for the operation of the Visibility 
Center, the establishment of goals and limits, and determination of contest winners, 
and duties associated with the maintenance and housekeeping of the center itself. 

The overall direction of the Visibility Center is a function of the managers 
responsible for the administration of specific programs within Procurement, Manu- 
facturing Operations, and Manufacturing Support. It is the function of Quality 
Engineering to provide the support to the user organizations in order to comply with 
their specific needs. 


MECHANICS OF OPERATIONS 


Data Collection 

Data are derived from inspection records, test results, and material review 
actions generated by workmanship related causes. 

Defect data used as a basis for determining process capability are collected 
from the inspection points and are screened to determine workmanship-caused defects. 
Workmanship defects are then equated either to units built, earned standard hours or 
to whatever base provides the most suitable common denominator. If the process is 
large enough, it may be divided into several groups or team units, and direct com- 
parisons can easily be made. However, if the processes are different and the com- 
plexity has wide variations, a suitable common denominator must be derived and an 
adjustment factor utilized to account for the variance in complexity. Process 
capability established through workmanship history is the basis of the Quality 
Visibility and Awareness Programs through stimulation of team competition. Quality 
awareness is promoted in a subtle manner through recognition derived through visi- 
bility and quasi-promotional programs which are discussed later in this paper. 

Defects that caused problems in which material review actions were necessary 
are utilized in employee performance measurement. The expense incurred, because the 
hardware is not reworkable to the drawing, is also highlighted in order to effect 
corrective action when needed and to emphasize the cost impact created by catastro- 
phic defects. 

As a measure of product quality, attribute test data are collected in terms of 
yield. Yield is preferred to rejection rate solely on the basis of presenting a 
positive profile. Yield data is used to signify trends and key corrective actions 
if indicated through tracking of specific assemblies or classes of hardware, i.e., 
continuity testing versus functional testing. 

Data Display 


Data are displayed in the Visibility Center that reflect trends in workmanship 
effectivity. Defects are plotted against unit gain with statistically determined 
control limits coupled with an adjustment factor to force improvement. Penetration 
of the control limit triggers problem analysis, identifies cause, and delegates cor- 
rective action requirement with effectivity commitment. 

This is a means to assure positive response to an undesirable trend. Effect- 
iveness of corrective action is accepted only through a positive reversal of the 
process trend. 

Data relative to test results, receiving inspection activity, material review 
board action, and scrap and rework cost impact are also maintained and displayed in 
the Visibility Center for use by the concerned responsible managers. See figure 1. 

MANAGEMENT PARTICIPATION 

The primary element to the success of the Quality Visibility and Awareness 
Program is the support and participation of all management levels. The support of 
top management is essential in assuring response and participation of all levels of 
management. 



Figure 1. 

Management participation usually occurs at three levels. Firstly, there is a 
daily review of individual operator performance by first-line supervision which is 
accomplished by review of the inspection sheets. The review serves a twofold pur- 
pose, namely, it keeps the supervisor informed of the types of defects and problems 
incurred, and provides an assessment of each operator's performance. 

s econdly, midcHe management and first-line supervision conduct weekly meetings 
in the Visibility Center to review quality status and to discuss corrective action 
commitments. 

Finally, weekly review of quality status and a follow-up of missed corrective 
action commitments are functions of upper level management. 

QUALITY VISIBILITY CENTER PROCEDURES 


Awards and Recognition 

. Manufacturing operations are divided into competitive units designated "Manufac- 
turing Quality Achievement Units". These units perform either as sub-groups within 
their assigned work centers or as units representing a unique work cost center or 
process. Each unit is in competition with all other units, for the purpose of im- 
provement recognition. Each unit has a visibility chart on display in the Visi- 
bility Center as well as a three-quarter size version on the unit supervisor's desk 
in the work area. 


The program is divided into monthly periods; the periods end with the last full 
work week of each month. Personnel comprising the "Unit" include the supervisor, 
employees who have participated within the unit for a least three weeks of the com- 
petition period, and material control support personnel. The unit supervisor is 
responsible for determining eligibility. All units are eligible for a recognition 
award in each competition period. 

The program administrator is responsible for providing adequate traceability on 
achievement unit visibility charts to identify unit achievement. 


The monthly award selection is based on the highest percentage of IMPROVEMENT 
(reduction) of defect rates. A degree of complexity factor based on past perfor- 
mance is determined and is used to adjust the performance ratios between units of 
dissimiliar processes. 


AWARENESS 

In order to promote employee Quality Awareness, the following methods have been 
utilized: 

-A number of illuminated signs are placed in strategic locations throughout 
the plant. Each sign displays different quality messages throughout the year. 

-Each monthly publication of the company organ carries a quality slogan as well 
as a photograph of the monthly award ceremony complete with story and names 
of the recipients. 

-The company pay envelopes are utilized periodically to convey messages and 
pamphlets on quality to each employee. 

-Special awards and citations for exemplary performance are given with suitable 
recognition in the company organ. 

-The trophy awarded the monthly Quality Achievement Award winners is displayed 
in the work areas of the winners. 

-An Honor Roll of past award winners is prominently displayed in the Visibility 
Center. 

Data Collection 


Workmanship performance is measured through a count of operator caused defects 
which are management controllable, based either on units built, chances for defects 
(complexity), or defects per earned standard hour. 

For competition purposes, inspection decisions are considered final; however, 
each of the area supervisors has appeal channels by daily review of inspection 
reports. 

Defect Charting 

Defect rates and material review actions are plotted to reflect performance 
levels at the lowest practical process level. 

Simply, inspection results are plotted on modified p-charts with respect to 
process areas. 

There are several significant areas used in the Northrop DSD visibility con- 
cept. First, a modified control limit is used, based on the average established by 
the process with a committed improvement factor included. 

Second, a weighted average is plotted, where the weighting is based on datum 
age and portrays the greatest significance when recent. This is a smoothed average 
and will be explained later in greater detail. 

Third, in the usual applications, the limits and defect averages are recorded 
and maintained in the office or in reports reviewed only by interested management 
and quality control staff. 

Computation of Control Limit Lines 

The control limit line is a mathematically established performance standard 
that, if penetrated or jeopardized by the cumulative average, triggers problem 
analysis and appropriate corrective action. Control limit lines are established on 
the basis of the following formula to determine the natural process limits: 

Control Limit Starting Point = 

As a matter of preference and control, a two-sigma limit is used as the start- 
ing point. The line slope defines the percentage of imposed improvement. 


Control limits are subject to periodic adjustment based on the most recent per- 
formance. If the adjusted limit start point is less than the end of the previous 
limit, the same percentage of improvement for the slope will be used for the new 
period. If the adjusted limit start point is greater than the end of the previous 
limit, the existing limit line will be extended horizontally for the next period. 

An exception to the rule is a management prerogative that permits raising of 
the limit based on significant uncontrollable influence on the process such as skill 
level changes, etc. 

Control limits are not used to indicate an acceptable performance, but as a 
goal to stimulate improved performance and as a problem analysis triggering function. 



COMPUTATION OF TREND LINES 

Computations of trend lines for this purpose are accomplished by using a moving 
average and a technique called "triple smoothing". This is a standard process that, 
when used on a computer, uses much less computer space than ordinary moving average 
computations. 

The outstanding advantage of this triple smoothing method of computation is 
that more emphasis is placed on recent data and less emphasis on older data. This 
tends to suppress random variations, which is not the case with ordinary averages 
which consider all data equally. 

Weight functions and response characteristics of ordinary averages comparable 
to triple smoothed averages, based on a six-unit ordinary average, are the following: 

Ordinary Average 

A single large datum will cause a sudden increase when it is Included and also 
a sudden decrease when it is removed. 

The effect on the average by the inclusion of a large datum may partially cor- 
rect or reverse the average when removed. 

Any given datum is given maximum weight when included, which is maintained 
until removal, when the weight reverts to zero. 

Triple Smoothed Average 

A single large datum entry will cause a smaller sudden increase when included 
and no sudden change from then on. 

There are no cancelling or reversing effects. 


A given datum is initially given a reduced weight and then an increased weight 
for the next two time units. The weight effect slowly decreases as the datum age 
increases, approaching but never reaching zero. 

Smoothing averages are computed by recursion. Initial estimates are used to 
start the computing process. The estimates can be ordinary averages, etc. For this 
application, ordinary averages of six units of data are used to begin the process. 
However, the initial estimate is satisfactory, as inaccuracies will be insignificant 
beyond six units of recursive updating. 

Triple smoothing uses three computations, called triple pass smoothing. A, A' 
and A" are averages from the first, second and third passes respectively; the sub- 
script "o" denotes the old average, the subscript "n" denotes a new average and X 
is a new datum to be included; "a" is the weighting factor. n 


A n * A o + a < x n - A o> 

A 'n= A 'o +a < A n- A 'o> 

A "n * A ”o + a < A 'n - A “o> 

The old single pass average A 0 , the old double pass average A' 0 and old triple 
pass average A" 0 must be maintained between updates. A" n is used for charting or 
comparison purposes. The constant "a" has been selected as 0.5 to provide the 
desired response. 

When "a" is 0.5, the triple pass computation can be reduced to the following: 
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PROBLEM IDENTIFICATION 

The Quality Visibility Center analyst identifies problems requiring corrective 
action based on analysis of defect causes. The analysis is focused on work centers 
where an unfavorable sequence of data points either approaches or penetrates the 
control limit. This condition demands an action item requiring corrective action 
and a committed effectivity. Confirmation of problem resolution is determined by a 
favorable trend reversal. 

Manufacturing supervision may identify problems through their review of Produc- 
tion Inspection Reports, area surveillance, or from knowledge of their respective 
work areas. 


CORRECTIVE ACTION 

Each problem requiring correction is placed on an Action Item chart, relative 
to the work center affected. It is keyed to the Defect Trend chart with a colored 
indicator. Colored indicators reflect the problem, the area of responsibility, and 
the status. The Action Item chart defines the problem, the corrective action, and 
the committed effectivity. 

The Corrective Action engineer effects corrective action by issuing a Notice of 
Corrective Action Requirement (NOCAR) assignment or placing the problem on the 
agenda of the Corrective Action Board (CAB) as necessary. The NOCAR is an instru- 
ment to assign responsibility for action and commit effectivity. The CAB is a board 
made up of the responsible managers with the authority to Implement corrective 
action throughout the organization. 



The Visibility Center coordinator follows up on all missed effectivity items, 
to assure that the next higher level of management is aware of the problem, and 
commits for corrective action. 


In the event of a 
submits the problem to 
managers. 


second missed effectivity, the Visibility Center coordinator 
the Corrective Action Board for review by the responsible 


AVERAGE COMPARISON: ORDINARY VS SMOOTHED 



Figure 3. 
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MEASUREMENT OF SUBJECTIVE VARIABLES 


Robert ¥. Hains, Alcan Canada Products Ltd., 
Kingston, Ontario 


Unambiguous definition of product characteristics is essential for specifica- 
tion of process control limits, monitors, feedback, control and corrective action 
systems. Most engineering data are measurable with physical instruments assisting 
establishment of reliable processes. 

In many instances product characteristics are assessed by response to sensory 
stimuli, e.g. taste, odour, appearance. This paper considers methods of measurement 
of these subjective quality factors. Discussion is restricted to appearance assess- 
ment although the underlying principles are usually applicable to other sensory 
measurements. 


THE PROBLEM 

In the absence of definition of appearance by physical metrics quality control 
relies upon a mental perception of "good*' and "bad”. Although the reject criteria 
may be supported by illustrative samples, mental interpretation of the standards can 
vary with individuals dependent on the relative weightings applied to the many sur- 
face components contributing to visual sensation. 

Process evolution tends towards minimizing defect frequency; the ideal system 
having a near zero probability of producing non-conforming material. Processes 
falling short of this utopian state generally require inspection monitoring to de- 
termine whether the system is in control and to segregate non-conforming material. 

The disadvantages of this method lie in the cost, post-hoc nature of the control 
system and variability in human response. 

Disagreements can develop in which a customer determines that material is un- 
satisfactory while the supplier believes that is of normal quality. Both parties 
consider themselves a victim of a change in standard on the other’s part. A de- 
cision to improve quality in future shipments generates stress in the production 
system which now has to adjust standards to an ill-defined level. 

Two potential solutions are evaluation of instrumental methods of defining sur- 
face appearance or control of process so that the product automatically satisfies the 
end use. The problems involved in development of instrumentation will be considered 
together with experimental methods for assessing the interrelationship of process 
variables and visual appearance. Data collection from designed experiments is sim- 
plified by the existence of a physical measuring system but is not dependent upon it. 
The methods of analyzing sensory reactions will be discussed. 

DEVELOPMENT OF INSTRUMENTS 

The visual impression of a surface is influenced by a variety of conditions 
rather than one single fact. Displeasure reactions may occur when a single condition 
exceeds a threshold value, through non-uniform distribution of a surface factor or 
due to coexistence of several imperfections at intermediate levels. Therefore a 
basic criterion for a measuring system is capability of determining intensity and' 
distribution of several surface elements. This is far from a simple task requiring 
that each surface component is identified and a method of measurement evolved. Such 
data are valueless unless they can be correlated with human like-dislike judgments. 
Thus a psychological scale is also necessary. 

Psychophysics, the branch of psychology devoted to measurement of human response 
to stimuli, originated with the German psychologist Gustav Fechner (1801-1887). 
Fechner hypothesized that a psychological ratio scale could be developed based on 
discovering a threshold value below which a stimulus is not perceived and a unit of 
measure, equal to the minimum detectable change in stimulus intensity - referred to 

as "just noticeable difference". Manning & Rosentahl^have published an elementary 
text describing the concepts and principles expounded by Fechner. 
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Much of the early work was devoted to assessing response to monatonic variables 
which were measurable instrumentally, e.g. discrimination between light intensity or 
difference between two weights. The problem becomes more complex when several factors 
contribute to value judgments. The difficulties in scaling under these conditions 

are discussed in the literature, for example Torgeson^, Coombs Edwards 

Swets^ and Galanter^, leading to the conclusion that psychological scale develop- 
ment is a very lengthy and costly process. 

The feasibility of economic development of instruments to measure appearance is 
low considering the dual difficulties of constructing metric devices and psychologi- 
cal scales. Thus the best instrument currently available is human - the inspector. 

VISUAL INSPECTION 

A trained inspector has many attributes of the ideal measurement and contour 
system, being equipped with a detector system, the eye; an interpretive component, 
the brain, capable of storing, comparing and making decisions based on the signals 
received from the eye; a feed back system either auditory or mechanical and ability 
for removal of substandard material. Deficiencies of the inspector are lack of con- 
sistency in decision and occasional failure to respond to a defect signal. 

Jerison and Pickett^, following work by Holland^ suggest that two pro- 
cesses are involved in an inspection task, a decision to observe and a decision to 
respond to a signal. Thus attention span and response decision determine inspection 
efficiency. 

Vigilant research is frequently interpreted in terms of Signal Detection Theory 
(SDT). Mackworth^ 10 ) presents a comprehensive review of the work in this field while 
Baker discusses SDT application to analysis of Q.C. Performance. 

In any inspection task one of four states exist: 

(1) Defect present, inspector rejects - a hit 

(2) Defect present, inspector accepts - a miss 

(3) No defect present, inspector passes - a hit 

( 4 ) No defect present, inspector rejects - a false alarm 

SDT strives to explain the occurrence of hits and false alarms. 

( 12 ) 

Tanner and Swets v ' suggest that a stimulus impinging on a sense organ changes 
the average value of some quantity which is ranging randomly within the brian. This 
neural process has a mean of zero when no signal occurs, and a non-zero value when a 
signal is received. Oscillation in the neural system will occasionally cause a 
critical non-zero value to be exceeded in the absence of a signal, giving rise to a 
false alarm. If the neural system is close to its low point, when a signal is re- 
ceived, the accompanying system change may be insufficient to exceed the critical 
response value, giving rise to a miss. 

Under the hypothesis that neural oscillations are normally distributed and the 
addition of a signal causes only a displacement of the mean, the model can be por- 
trayed as two normal distributions with equal variance and means separated by a dis- 
tance equal in magnitude to the intensity of the stimulus expressed in terms of stan- 
dard deviations. If the separation of the means is such that the distributions over- 
lap, misses and false alarms will occur. The probability distribution of hits, 
misses and false alarms are computed from the degree of overlap. 

The probability model, thus developed, implies that an individual has an innate 
sensory discriminating power, a function of the standard deviation of neural system 
oscillation. This would suggest predictability of hits and misses, in overlapping 
distributions, with the central tendency of the pass-reject criterion being the 
intersection of the distributions. 

(13) 

Swets et al v ' suggest that the monitoring task involves a decision in addition 

to discrimination, yielding two values d 1 and B. Discrimination (d 1 ) is a function 
of the separation of means between the noise and noise plus signal distribution and 
measures sensory capability. The decision involved in establishing a signal cri- 
terion (B) is a measure of the caution of the observer. The value of B is influ- 
enced by many factors. Smith'"^ describes how a single word from an authority 



figure can change the proportion of product passing inspection. Criterion is a vari- 
able reflecting many influences, particularly those related to perceived rewards of 
making a ’'correct" judgment and the cost of making errors. Criticism for releasing 
defective material usually leads to an increase in the false alarm rate. 

Baker summarizes factors enumerated by Colquoun^ 1 "^ and McKenzie^ 1 ^ which 
affect inspector decision but omits the vital aspect of expectancy. 

There is ample evidence that the expectancy of an observer influences his 
stimulus response, as illustrated by the following case histories. 

Case History I 

A process, in early stages of evolution, consistently exhibited a defect rate 
in the range 8 to 12 percent. After the cause was isolated and corrective measures 
instituted, a marked drop in defect level occurred but exceeded the level predicated 
prior to implementation of the changes. 

Investigation determined that the majority of defects were restricted to lots 
examined by a particular inspector. Re-examination of rejected material by the in- 
spection forman showed that the majority of rejects actually conformed to standard. 

The inspector was surprised that the foreman disagreed with the original dis- 
position decision and thought his decision was right "because we always get about 10 
percent defectives with this material." 

Whatever connotation is placed on the mental process motivating the inspector’s 
action it is evident that the criterion applied was based on his expectation that one 
tenth of the material was unacceptable. 

Case History II 

The actors are four engineers and technicians, each having extensive experience 
in assessing surface appearance of aluminum extrusions. 

A test was being conducted to assess the validity of the hypothesis that above a 
threshold level increase in an impurity level caused progressive deterioration sur- 
face finish. The threshold was not defined accurately but predicted to be within a 
specified range. 

The experimental design (discussed later) consisted of extrusion of groups of 
ingots containing levels of impurity ranging from close to zero to an order of magni- 
tude above the upper bound of the expected threshold. Extrusion order of each group 
was randomly assigned, each experimental group being interspersed with a like size 
batch of control ingot. 

Subsequent analysis of samples from the experiment showed no significant dif- 
ference between the surface finish of any conditions tested. However the behavior 
of the observers during the test is instructive. 

Extrusion sequence in the early stages of the experiment was: 

(1) Control 

(2) Impurity in threshold range close to lower bound 

(3) Control 

( 4 ) Impurity below lower bound of threshold 

( 5 ) Control 

(6) Maximum impurity content 

As the test proceeded each observer noted that group 2 was essnetially the same 
as group 1, leading to the conclusion that the threshold level had not been reached. 
Group U was similar to control groups, an expected result. While group 6 was being 
extruded the observers became more attentive, heads moved towards the product which 
received close scrutiny. 



Subsequent questioning showed that each observer went through similar 
thought processes illustrated by the following sequence table. 


Stage 

Time 

Reactions and Thoughts 

1 

Immediately prior to 
extrusion of group 6 

We will now see a marked de- 
terioration in quality 

2 

Extrusion of first ingot 
group 6 

Looks same as control - probably 
too early to see anything yet - 
probably show up on next ingot 

3 

Extrusion of second 
ingot group 6 

Still looks good - deterioration 
must be less than expected, must 
look harder 

4 

Extrusion of subsequent 
ingots 

As I look harder I can see something 

I did not see in previous ingots. 

I think it is worse - but not as 
bad as I thought it would be 


While in reality there was no significant difference between group 6 and pre- 
ceding groups, expectation of a deterioration caused more intense scrutiny to the 
extent that features, overlooked in previous material, became noticeable and classi- 
fied as defects. This is a clear example of expectancy changing the decision 
criterion. 

Maintenance of a consistent decision criteria relies upon retention of a mental 

(17) 

picture of a visual standard. Thomas' ' found that experienced inspectors acquired 
an image of good material against which defects were highlighted. However it is 
contended that the mental image is subject to variation. The following adds credi- 
bility to such a hypothesis. 

Case History III 


In two series of experiments control ingot, selected randomly from the same 
population, was compared with experimental material, treatments applied to which 
differenced between the series. Three experienced observers witnessed each run in 
both experiments. 

Prior to the final examination of samples all observers agreed that in the 
second series the surface appearance of the material produced from the control ingot 
was significantly worse than observed in the first series. When the coded samples 
were compared it was clear that the control material from both sets of experiments 
came from the same appearance populations. 

The difference between the observers' impressions and reality probably resulted 
from differences in contrast between the control and experimental material in the 
two tests. In a paired comparison test the control material was invariably better 
than the experimental material in test 1 but in test 2 there was considerable overlap 
in the appearance distributions of the control and experimental sets. 

These observations suggest that contrast in appearance in the first test re- 
sulted in suppression of signals emanating from the control group. Because the sur- 
face appearance of all material in the second group was very similar, more intense 
scrutiny was required to differentiate between the control and experimental material. 
This caused unconscious increase in observer attention revealing small imperfections 
in the control group unnoticed in the first series, leading to the conclusion that 
the control had a worse finish in the second series than in the first. 
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It can therefore be reasoned that an inspector is likely to accept substandard 
material which is close to specification limit if the preceeding material contained 
gross defects* Conversely, conforming material may be rejected if preceded by high 
quality. 

The post-hoc nature of inspection, coupled with inspector variability and cost, 
are driving forces for automatic control of factors leading to rejectable material. 
Identification of these parameters are best achieved through designed experiments; 
however such methods present challenges in both determination of the experimental 
format and in measurement of response - surface appearance. 

EXPERIMENTAL DESIGN 

Determination of the format and size of an experiment necessitates consideration 
of characteristics peculiar to the test procedure, the method of obtaining data and 
the type of result being sought. Once these factors have been established standard 

references on experimental design such as Davies Mendenhall^ ^ and Cochrane and 
( 19 ) 

Cox ' are useful in determining the most efficient manner of obtaining the neces- 
sary data. 

In many metal working processes tool-metal interaction causes formation of 
boundary layers which influence appearance. Frequently the nature of this film and 
consequently the quality of surface of the formed material is influenced by charac- 
teristics of the material being processed. Change in the material passing over tool- 
ing may not be accompanied by an immediate conversion of the boundary layer which may 
pass through transition phases before the characteristic layer of the new material is 
fully developed. Thus sufficient material must be processed to ensure conplete 
transformation before appearance of a patch is judged. 

Time dependent changes influence product appearance in many processes due to 
factors such as tooling wear. Where such trends are known to occur or are highly 
probable, the sequence of variables being evaluated should be balanced with respect 
to time. 

The method of evaluation must be designed to minimize errors resulting from con- 
trast effects and observer expectancy discussed previously. The first of these con- 
siderations dictates that data should be obtained from samples rather than from ob- 
servation during the experiment. 

To minimize expectancy judges must be ignorant of the treatment employed which 
is best achieved by presentation of coded samples in random order. 

There are four basic methods of judging surface finish: magnitude estimation, 
rating by standard, ranking and paired comparison. The first two of these are highly 
error prone. 

In magnitude estimation the subject is presented with samples one at a time and 
is asked to rate it on a numerical appearance scale. We have already seen that the 
magnitude of any judgment will be influenced by the items just previously examined. 

(2i) 

In addition, Miller' 1 concludes that the average subject can categorize correctly 
into somewhere between 5 and 9 categories irrespective of the sensory dimension of the 
experiment • 

Rating by standard entails use of a number of standards spanning the range. The 
subject is presented with samples one at a time and asked to identify the pair of 
standards which bracket his subjective reactions. One difficulty with this procedure 
is that if the difference between adjacent standards is small enough to ensure ade- 
quate separation of the items into homogenous groups, different observers are certain 
to disagree with the order of the standards. 

Ranking requires that the panelists place all samples in a progressive array 
from worst to best. This method has the advantages that the subject conpares samples 
rather than relying on a mental image. In addition the rating is determined by the 
subject. 


The number of comparisons required for positioning a sample are few in the 
initial stages of ranking but increases rapidly as the process continues. Thus 
ranking of large numbers of samples is tedious and time consuming. The time taken 
by five observers to rank 72 samples from an experiment varied from 1^0 minutes to 
260 minutes. Under such conditions vigilance decrement, first enunciated by 
( 22 ) 

Mackworth v 1 , is highly probable leading to misranking of some samples. The result- 
ing reduction in accuracy of the data is unlikely to have serious consequences in 
experiments containing several replicates of each factor. Displacement of a sample 
from its "correct” rank is probably small if the ranking of several inspectors show 
highly significant coefficients of concordance and rank averages are used. 

Data from ranking tests can be used for analysis of variance providing the 
dangers of applying parametric tests on non-parametric data are appreciated. Output 
of such analyses can be used to determine which factors influence the measured re- 
sponse but cannot be used to obtain estimates of magnitude of differences. This 
metric can however be estimated by paired comparison. 

In paired comparison testing two samples are presented and the subject asked to 

(23) 

express preference for one over the other. Doehlert ' discusses various strategies 
of testing which range from evaluation of all possible combinations of pairs to de- 
signs in which each judge sees fewer pairs arranged so that the combined data for all 
judges include every pair an equal number of times. Several techniques are available 
for data analysis which are referenced by Doehlert. 

The major advantage of paired comparison is that the subject does not have to 
rely on previous responses or scoring systems. In addition, each response requires 
only one decision in a forced response situation. This increases the probability 
that "wrong" results occur only when samples differ by an amount which is not much in 
excess of the discriminating power of the subject. 

On the negative side, a balanced design involving comparison of all possible 
pairs requires n(n - l)/2 observations to generate one data set. Since most evalu- 
ation schemes presuppose a number of judges, each judge performing the test a number 
of times, the method becomes cumbersome unless the total sample size is small. 

Undoubtedly there are situations (e.g. taste testing) in which these methods 
are justifiable. However in appearance evaluation such detailed analysis is fre- 
quently unwarranted because the finished product does not receive anywhere near the 
same degree of scrutiny as is required in making a decision between txvo samples. 

An ideal solution is to combine the simplicity of the paired comparison evalu- 
ation but reduce the amount of testing necessary to obtain a reliable result. 

Doehlert suggests method in which three or more items at a time are evaluated thus 
presenting a system which is a hybrid of the pure ranking system and paired compari- 
son. 


In many industrial situations the problem falls into one of two categories: 

(1) If a given process change is introduced will there be a noticeable 
improvement in quality justifying the costs of the modification. 

(2) If a cost reduction scheme is introduced, will it result in a 
noticeable deterioration in quality. 

In both of these situations the word ’noticeable 1 implies a significant change 
in the distribution either by variance reduction or displacement of the mean. The 
former can result from tightening of a control range, the latter by alternation in 
level of process parameters. Appropriate experimental designs would enploy a control 
which represents current methods and experimental factors using either tighter con- 
trol range or alternative level of a parameter. In the case where the experiment 
assesses the effect of tolerance reduction, the level of the parameter for the con- 
trol ingot is selected to be outside the proposed limits. 


If we assume that a quality attribute (e.g. appearance) of the conceptual 
population is normally distributed, with variation resulting from changes in many- 
process factors, then strictly we need to know the magnitude of the mean and variance 
of the conceptual populations of both the current and proposed processes. This leads 
to the complex analysis previously considered. However, a designed experiment is 
carried out over a short time span and unusual measures are employed to maintain 
other variables as close as possible to their mid range positions, from which it 
follows that control material will tend to be close to mean and the variance will be 
much less than the conceptual population of current production. A similar situation 
exists between the mean experimental group and the anticipated mean of the conceptual 
population of the amended process. It is hypothesized that under these circumstances 
a significant change in quality occurs only if there is an "always noticeable dif- 
ference" between the populations. The condition that the degree of scrutiny of the 
final product is much less than that in a paired comparison test is essential to this 
assumption. 

The criterion of always noticeable difference can be evaluated using paired 
comparisons in which all control are compared xri.th all experimental groups. Omitting 
within groups comparisons reduces the number of pairs evaluated for one data set from 

n(n - l)/2 to n^/ 4 . 

The number of observations needed for a decision can be reduced further under 
the hypothesis that as a result of the control imposed, each group has approximately 
the same variance. Under this hypothesis an upper limit to the number of pairs re- 
quiring comparison can be generated, based on the probability that a specific degree 
of distribution overlap will be detected for a particular sample size. 

A distinct advantage of the proposed method is that evaluation is aborted if it 
becomes evident that the hypothesis of an always noticeable difference is invalid. 
Thus prolonged testing occurs only when the result is likely to be useful industri- 
ally. 


SUMMARY 

Product acceptability is frequently determined by human response to sensory 
stimuli. Absence of industrial measuring devices prevents clear definition of 
standards and application of automatic control systems. 

Development of a measuring system reflecting human response to visual appear- 
ance requires instrumental techniques to assess the density and distribution of the 
myriad of surface components influencing like-dislike decisions. These need corre- 
lating with material rated on a sensory scale. The complexity of this task makes 
industry reliant on assessment by inspectors who are subject to vigilance decrement 
and changing decision criteria. 

Designed experiment to identify process parameters which adversely affect 
sensory related quality are hampered by absence of physical methods of measuring re- 
sponse. Discussion of evaluation methods suggests that paired comparison can be 
used, even when the number of samples evaluated is large, if the decision criteria 
is an "always noticeable difference." 
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DETERMINISTIC AND PROBABILISTIC FRACTURE MECHANICS 


Sherrell D. Manning, Senior Structures Engineer 
James M. Norton, Senior Operations Analyst 
General Dynamics /Fort Worth Division 
Fort Worth, Texas 76101 

ABSTRACT 

"Deterministic" and "probabilistic" approaches for predicting 
crack growth in high performance aircraft are examined. Each approach 
is described in a simple conceptual manner for six basic elements of 
fracture mechanics. Concepts are reviewed and the potential of the 
"probabilistic" approach is discussed. 

INTRODUCTION 

Damage tolerance and durability requirements are now being imposed 
by the Air Force on current and future metallic airframes [l,2]l. The 
effects of material and manufacturing flaws, such as, cracks, scratches 
and voids must be evaluated at three levels: (1) preliminary design, 
(2) final design and (3) service life tracking. Efficient and reliable 
analytical methods are needed to evaluate the behavior of flaws under 
expected service conditions. Structural performance, life and owner- 
ship costs depend on the flaw growth prediction methods used and fleet 
service life management activities. 

A deterministic approach is currently used to predict level of 
damage (i.e., crack size) and life for aircraft structures [3,4], The 
total crack length is predicted using an initial crack size and a 
linear accumulation of crack growth on a cycle-by-cycle basis. This 
method is inefficient and costly to implement because crack growth is 
predicted and accumulated for each load cycle in the load spectra. A 
more streamlined methodology is needed for predicting crack size for 
different materials, design concepts and load spectra and for tracking 
the cumulative damage of each aircraft in the fleet. 

Methods have been proposed for simplifying crack growth analysis 
[5,6,7]. Progress has been made but further improvements are needed. 

Current methods for predicting crack size are deterministically 
oriented although statistical and probabilistic principles are also 
used. It is called a "deterministic" approach because the variables in 
the analysis are considered as single values or constants with zero 
variance. A single prediction of crack size results for each set of 
values input. 

Recently, considerable interest has been shown in structural reli- 
ability and probabilistic approaches in structural design [8]. A "pro- 
babilistic" approach considers the governing parameters as random vari- 
ables. Each variable is considered as a distribution of values with a 
mean and variance. The feasibility of a probabilistic approach for 
predicting crack size in aircraft structures needs to be examined. 
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The objective of this paper is to study the feasibility of a pro- 
babilistic approach for predicting cumulative crack growth in aircraft 
structures. This objective is satisfied as follows. The deterministic 
and probabilistic approaches are conceptually described and discussed 
for each of the following crack growth elements: 

o Initial Flaw Size 
o Fracture Toughness 
o Crack Length versus Time 
o Crack Growth Rate 
o Load Spectra 
o Crack Growth Accumulation. 

The implementation of the probabilistic approach is discussed and the 
potential of this approach is assessed for near term applications. 

PHILOSOPHICAL CONCEPTS 

Deterministic and probabilistic approaches are characterized for 
six basic elements in Figures 1 and 2 respectively. Each element is 
conceptually described below. Many of the concepts are presented in 
an ideally simplistic form to aid understanding and comparisons. Basic 
definitions applicable to both approaches are given in the determinis- 
tic approach discussion. The term "approach", as used in this paper, 
refers only to the philosophical point of view reflected in the method- 
ology and not the detailed procedures for predicting crack growth. 

Deterministic Approach 

The deterministic approach refers to methods for predicting level 
of damage (i.e,, crack size) and life in which all input data are con- 
sidered as discrete values. Thus, a single value prediction is obtain- 
ed for a given data set. 

Crack growth variables, such as, flaw size, fracture toughness, 
and crack growth rate are described by assumed probability density fun- 
ctions (PDF). These functions mathematically describe the distribution 
of possible values for each variable. Suitable functions are selected 
using available test results, prior experience or intuition. For crack 
growth analysis, one value is selected for each variable from its dis- 
tribution of possible values. Selected values are considered as "worse- 
case" and they acknowledge that the distributions were estimated from 
finite sample sizes. For example, the 90th percentile estimated with 
95% confidence. This idea applies to the items in Frame A through D 
in Figure 1. 

Initial Flaw Size - An initial flaw size is required for critical areas 
of the structure to start the crack growth analysis. The distribution 
of initial flaw sizes is depicted in Figure 1, Frame A. One initial 
flaw size is selected from the distribution. Initial flaw sizes for 
damage tolerance are specified in MIL-A-83444 [1], Large flaws grow to 
their critical length faster than smaller potential flaws when both are 
subject to the same stress history. This assumption is basic for the 
deterministic approach. 

Fracture Toughness - This material property (K c ) measures the material’s 
resistance to fracture. Kc is used to screen candidate materials and to 
predict the material performance of full-size structure using simple 
test results. Structural failure is predicted using Kc, applied 
stresses and probable flaw size. K c is a random variable. For analysis 
purposes, one value of K c is selected from the distribution of values to 
compute the critical crack length (Figure 1, Frame B). 
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FIGURE 1 Elements Of The Deterministic Approach For Predicting Crack Growth 


Crack Length versus Time - Crack growth data, such as crack length (a) 
versus time (N) , are developed from constant amplitude tests. Tests are 
performed for a given material using replicate specimens. Results are 
used to evaluate crack growth rates. The crack growth performance of 
identical specimens with the same initial flaw size varies although 
specimens are subjected to comparable constant amplitude testing (Figure 
1, Frame C). For example, curve 1 shows faster crack growth than curve 
2 in Figure 1, Frame C. For analysis, only one "a" versus'V curve is 
selected for a specified probability and confidence level. 

Crack Growth Rate - Incremental crack growth is predicted for different 
materials and loading situations using this information. The crack 
growth rate, da/dN, is derived from a (crack size) versus N (time) test 
results. Log-log plots of da/dN versus AK are depicted in Figure 1, 
Frame D. The plotted data is bounded by a confidence interval. One 
da/dN versus AK curve is selected for a given probability and confi- 
dence level for analysis purposes. 




















Load Spectra - Loads are defined using a load- exceedance curve. A load 
history, with a specified load sequence, is needed to perform the crack 
growth accumulation analysis (Figure 1, Frame E) . Current practice is 
to simulate the overall life on a flight-by-flight basis. Each flight 
in the load spectra includes a series of cycles that combine the deter- 
ministic and probabilistic events describing the type of mission. 
Deterministic events include takeoff and landing and certain maneuvers 
(such as cruise, climb, TFR, etc.) during each flight. Events such as 
gusts or rough field taxiing occur periodically. The number of times 
these events occur can be estimated but their position in the load 
sequence is determined in a probabilistic manner. An infinite number of 
load sets with different load sequences are possible to represent the 
service usage of all aircraft in the fleet. However, a specific load 
history simulation, that includes a specific sequence of loads, is used 
in the deterministic crack growth accumulation analysis. The aircraft 
usage severity is a random variable but the effect on the predicted 
crack growth accumulation is considered in a deterministic manner. 

The deterministic load history (with a specific sequence of loads) 
is transformed into a stress history for each critical area of the air- 
frame. This is accomplished by determining the relationship between the 
load history and the stress response of the structure in the critical 
areas . 

Crack Growth Accumulation - The crack growth is accumulated by adding 
to the current crack size a crack growth increment, which reflects the 
effects of the next load cycle (Figure 1, Frame F). Crack growth 
accumulation is computed using: 

n 

af = ai + X ( Aaj) (1) 

where, = initial flaw size 

Aaj = crack growth increment associated with the 
jth load cycle. 

Load interaction models account for load sequence effects on crack 
growth increments [9-11]. Thus, the predicted crack size is accumulated 
on a cycle-by-cycle basis assuming that crack growth is a deterministic 
process. 

Probabilistic Approach 

The "probabilistic" approach refers to methods for predicting the 
distribution of crack sizes (or damage) and structural life by con- 
sidering the governing parameters of crack growth as random variables. 
Variables, such as flaw size, fracture toughness, crack growth rates, 
loads and environment are considered as statistical quantities. Values 
for these variables are treated as statistical distributions with a mean 
value and variance. Results are presented in terms of probability of 
equaling or exceeding a specified value for a given data set. The goal 
is to predict the distribution of damage and life in a statistical for- 
mat so that the probability of reaching specified limits can be deter- 
mined . 

Initial Flaw Size - An initial flaw size distribution describes the 
population of potential initial flaws in the structure (Figure 2, Frame 
A). Whereas the "deterministic" approach considers a single "worse 
case" flaw size for analysis, the "probabilistic" approach treats init- 
ial flaw sizes as random variables. 
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Crack Growth Accumulation - The current crack size is considered a 
random variable (Figure 2, Frame F). Symbolically, the crack size 
can be expressed as: 


<P( af) = f(ai) + X F ( Aa j) (2) 

j=l 

where: <^(af) = distribution of current crack size 

f(a^) = distribution of initial flaw size 

F( Aaj) = distribution of crack growth increment 
which depends upon such things as the 
crack growth rate, load history and 
environment. 

IMPLEMENTATION OF PROBABILISTIC APPROACH 

The purpose of this section is to discuss, in general terms, the 
implementation of the probabilistic approach for predicting crack 
growth. The probabilistic approach for predicting crack growth is con- 
ceptually described in Figure 2 for six basic elements. These concepts 
are described in a philosophical manner without regard to their imple- 
mentation in a detailed crack growth analysis procedure. 

The distribution of crack sizes after a given service time, or 
equivalently the distribution of service time to reach a given crack 
size, is the goal of the methodology regardless of how many of the 
basic elements are allowed to be random variables (Figure 2, Frame F) . 
The problem is to derive these distributions for selected aircraft con- 
trol points using the expected service conditions (loads and environ- 
ment) and predicted structural response. Once these distributions have 
been defined, probabilistic principles can be used to predict: 

o the probability that the cumulative crack size is equal 
to or greater than a specified crack size at a given 
time period, i.e., 

Pr(a 2 a 4 = Nf) (3) 

o the probability that the life is equal to or greater 
than a specified value for a given crack size, i.e., 

Pr(N > Nf| a . af ) (4) 

Two possible methods for deriving the needed distributions are: 

(1) Analytical Derivation [12] and (2) Monte Carlo simulation [13] . 

Analytical Derivation [l2] 

Theoretically, the crack length distribution can be analytically 
derived as a function of time. A simple example illustrates the essen- 
tial features of the analytical derivation without introducing the 
mathematical details (Figure 3). There are two basic problems: (1) 
characterize the initial crack length distribution and (2) determine 
the crack growth relationship as a function of time. First, the init- 
ial crack length is assumed to be a random variable. The functional 
form of the distribution of initial crack lengths is assumed (Normal, 
Weibull, Log-Normal, etc.). Parameters for the distribution are esti- 
mated from available data. 



A mathematical expression, F(N), is needed which describes the 
growth of an arbitrary crack as a function of time. F(N) is substituted 
into the initial crack size distribution to obtain the desired trans- 
formed distribution of crack sizes. The distribution of times to reach 
a given crack size is determined in a similar manner. F(N) depends upon 
the crack growth process and the derived equation yields a single value 
of F(N) for a given N value. Thus, the transformed distribution is 
obtained using a deterministic equation. 



FIGURE 3 Transformation Of Initial Crack Size Distribution 


Monte Carlo Simulation f 13] 

A distribution of values (PDF) is defined for each significant ran- 
dom variable in the crack growth process. One value of each variable is 
randomly drawn from applicable populations. These values are substi- 
tuted into a model or equation which characterizes the crack growth 
process. The result is a single value for a crack size at a given time 
or the time to reach a given crack size. One prediction is obtained for 
each data set. New data sets are drawn and the above calculations are 
repeated to establish the distributions needed to implement the prob- 
abilistic approach (Figure 4). 










Problems 


Several problems must be considered to implement the two methods 
discussed. This includes such items as: 

o Error and uncertainty 

o Lack of data to support analyses 

o Joint distributions 

o Proven model for crack growth process 

o Significant crack growth variables and their 
inter-relationships 

o How to apply simple data to complex structure 
Further discussion of these problems is beyond the scope of this paper. 

POTENTIAL OF PROBABILISTIC APPROACH 

Is it feasible to implement a '’probabilistic" approach for evalu- 
ating crack growth and also satisfy the Air Force's damage tolerance 
[l] and durability [2] requirements? If so, this approach should be 
responsive to the designer's needs at the preliminary and final design 
levels. It should provide design data, such as, allowable stress levels 
for different materials, design concepts, load spectra, and environment. 
The specified design allowables must satisfy the damage tolerance and 
durability requirements. Reliable results are needed quickly to meet 
schedule and cost requirements. Also, procedures should be reliable 
and efficient for evaluating the cumulative damage for fleet members 
under service conditions and for evaluating fleet maintenance and repair 
requirements • 

Using the above criteria, the "probabilistic" approach for pre- 
dicting crack growth, in which all parameters in the crack growth pro- 
cess are treated as random variables, is not feasible for near future 
aircraft applications. Presently, an approach that includes both deter- 
ministic and probabilistic features is the most promising. However, 
several issues must be resolved before the combined "deterministic/ 
probabilistic" approach envisioned can be implemented with confidence. 
First, a better understanding of the crack growth process is needed for 
different materials, design concepts and operating conditions. Second, 
the significant crack growth variables and their inter-relationships 
must be better defined so that emphasis can be placed on the proper 
variables and their statistical characterization. Third, better deter- 
ministic models are needed to describe the crack growth process. Such 
models are essential for implementing the combined "deterministic/ 
probabilistic" approach for predicting crack growth. 

Monte Carlo simulations would be time consuming for implementing 
the probabilistic approach. For this reason it would be too inefficient 
for the needs of preliminary and final design and for tracking the cumu- 
lative damage of fleet members. 

Mathematical transformations are promising for implementing the 
probabilistic approach. However, improved models and statistical char- 
acterization of the crack growth process are needed. 



CONCLUSIONS 


A classical "probabilistic" approach, in which all variables and 
the crack growth process are considered random, is not practical in 
terms of the present state-of-the-art. For near future applications, 
the deterministic approach, including the use of statistical and prob- 
abilistic principles, is the most promising for predicting crack growth 
in aircraft structures. The combined "deterministic/probabilistic" 
approach is promising but further development and understanding of the 
crack growth process are needed to implement it. It should be recog- 
nized that the deterministic approach can be generalized into an 
increasingly more probabilistic approach by regarding one or more of 
the crack growth parameters as random variables. 

The most efficient way to streamline the existing crack growth 
methodology with the present understanding, is through statistical char- 
acterizations of the load spectra and structural response. A better 
understanding of the mechanisms of crack growth is needed. This 
involves defining the significant crack growth variables and their 
relationships . 

Improved load-interaction models should be developed to replace the 
inefficient cycle-by-cycle models [9,10,11], New models should place 
greater emphasis on the use of statistical and probabilistic principles 
for characterizing crack growth and the load spectra. Statistical and 
probabilistic principles are valuable tools for evaluating the effects 
of variability. These tools should be fully exploited to further the 
crack growth analysis state-of-the-art. 

ACKNOWLEDGEMENTS 

Mr. Walt Paquette and Dr, Glen Lemon of the General Dynamics, 

Fort Worth Division and Mr. Jim E. Littlefield of the Vought Corporation 
made helpful suggestions. Mrs. Ina Rae Bell typed the manuscript. 

REFERENCES 

1. MIL-A-83444, "Airplane Damage Tolerance Design Requirements." 

2. MIL-A-008866B, "Airplane Durability Design Requirements." 

3. Wood, H. A., "A Summary of Crack Growth Prediction Techniques," 
AGARD Lecture Series No. 62, 1973. 

4. Wood, H. A., J. Gallagher, R. M. Engle and J. M. Potter, "Current 
Practice on Estimating Crack Growth Damage Accumulation with 
Specific Application to Structural Safety, Durability/Reliability," 
AFFDL-TR-75-32, January 1976. 

5. Gallagher, J. P., and H. D. Stalnaker, "Methods for Analyzing 
Fatigue Crack Growth Rate Behavior Associated with Flight by 
Flight Loading," Paper No, 74-367, Am. Inst. Aero, and Astro., 

April 1974. 

6. Barsom, J. M. , "Fatigue Crack Growth Under Variable Amplitude 
Loading in Various Bridge Steels," STP 595, 1976, pp. 217-235. 

7. Brussat, T, R. , "Rapid Calculation of Fatigue Crack Growth by 
Integration," STP 559, 1973, pp. 298-311. 



8. Lemon, G. H. , and S. D. Manning, "Literature Survey on Structural 
Reliability," IEEE Transactions on Reliability, Vol. R-23, No. 4, 
October 1974, pp. 263-266. 

9. Wheeler, 0, E., "Spectrum Loading and Crack Growth," J. of Basic 
Engineering, ASME, Vol. 94, Series D, No. 1, March 1972, pp. 181- 
186. 

10. Willenborg, J. D. , R. M. Engle, Jr. and H. A. Wood, "A Crack 
Growth Retardation Model Using an Effective Stress Concept," 
AFFDL-TM-71-1-FBR, AFFDL, W-PAFB, Ohio, January 1971. 

11. Johnson, W. S., and T. Spamer, "A User's Guide to CGR-GD, A 
Computerized Crack Growth Predicting Program," General Dynamics 
Report FZS-241, November 1976. 

12. Hogg, R. V., and A. T. Craig, Introduction to Mathematical 
Statistics , New York: Macmillan & Sons, 3rd Edition, 1970. 

13. Benjamin, J. R. , and C. A. Cornell, Probability Statistics and 
Design for Civil Engineers , New York: McGraw-Hill Book Company, 
1970, p. 124. 

LCS : 030:60:434 



QUALITY ASSURANCE OF METAL IN CAN MANUFACTURING 


King 6. Robertson, Manager of Metallurgy and Physical Testing 
Coors Container Company 
Golden, Colorado 80401 


INTRODUCTION 

The manufacture of two-piece beer and beverage cans by the draw-and-iron method 
has been commercially significant for only a little more than a decade. Today, it is 
a major manufacturing process involving some two dozen companies at nearly one hun- 
dred locations producing annually about 30 billion two-piece cans worth over 2.5 
billion dollars. 

Coors Container Company was an early entrant in two-piece can manufacture, and 
was a significant contributor to its rapid expansion. While such cans are made today 
in both steel and aluminum, Coors has worked commercially only with aluminum, and 
today has a yearly plant capacity of over 3.5 billion cans at its single Golden, 
Colorado, location. Coors 1 first two-piece can line for 11-ounce cans was designed 
to run at a speed of 1000 cans per minute. That same line now manufactures cans at 
over 1800 per minute, and the newest line is designed to reach speeds as high as 
3000 cans per minute. 

During the same decade of progress in speed of manufacture, the price of alumi- 
num stock for making cans has escalated from less than 30 cents per pound to more 
than 70 cents per pound, and coated metal for end making has risen from 45 cents per 
pound to nearly one dollar per pound. It is not surprising then that an effective 
quality system is essential to control and improve the costly material that feeds 
this high volume, sophisticated process. 

THE QUALITY SYSTEM 

The metallurgical quality system at Coors is dependent on two basic requirements : 
first, a thorough understanding, not only of the can making process, but also the 
fabricating process of the sheet; second, a close liaison with our supplier's tech- 
nical, sales, and quality personnel for rapid response to problems. 

A quick examination of the quality variables and attributes as they relate to 
mill fabricating operations will reveal the importance of those requirements. 

TABLE I. 


Mill Operation 

Can 

Stock Defect/Attribute 

1 . 

Mel ti ng 

1 . 

Composition, cleanliness, inclusions 

2. 

Casting 

2. 

Surface oxide, grain size, insoluble precipitate size 

3. 

Scalping 

3. 

Surface quality, slivers, tears, rolled-in-oxides 

4. 

Preheating 

4. 

Homogeneity, solid state transformations, formabil- 
ity, non-galling 

5. 

Hot rolling 

5. 

Gauge uniformity, edge cracking, earing, oxide pick- 
up, shape 

6. 

Anneal i ng 

6. 

Grain size, earing, oxide surface, properties, 
scratches 

7. 

Cold rolling 

7. 

Final gauge and variation, scratches, shape, flat- 
ness, rol led-in-dirt, oxide, etc. 

8. 

SI itting/inspecting 

8. 

Width, edge burrs, flatness, even coil wind, dirt, 
oil, edge cracks, gauge 

9. 

Pack/ship 

9. 

Correct weight/ footage, clean moisture free package, 
no shipping damage. 
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One look at the above list should make several things apparent. No amount of 
wishing at the can plant will make off-specification material work. Corrective 
action must be taken by the supplier at his mill. Rapid and accurate feedback from 
the user is the only practical way to achieve that result. This is extremely impor- 
tant when one adds the fact that in the pipeline at any given time is a supply of 
material worth several million dollars. 

How do we assure rapid and accurate feedback for corrective action? With 
people! Figure 1 is a diagram of the organization at Coors Container Company, detail- 
ing the Metallurgical and Physical Testing Department. 

Figure 1. Coors Container Company Organization (partial) 
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The Metallurgy Department has primary responsibility for metal in the following 
areas. 

I. Quality 

A. Specifications 

B. Vendor capability audits 

C. Incoming inspection 

D. Material/process audits 

E. Material traceability 

F. Vendor source audits 

G. Problem solving 

H. Material disposition 

II. Research and Development 

A. Qualification 

1. New suppliers, locations 

2. New practices, processes 

3. New alloys 

B. Evaluation 

1. New forming concepts 

2. Minor practice or process changes 

3. Material processes 

C. Technical liaison 

D. Supply and Demand (metal) 

III. Physical Testing 

A. Metals and coated metals 

B. Division support 

C. Corporate support 







THE TEAM CONCEPT 


How does the organization in Figure 1 accomplish the above mission, including 
the rapid feedback to implement corrective action on defective materials? It does 
so as a member of a team, or rather of several teams, which in concept extends 
throughout the Coors organization and into the organization of each major material 
supplier. A typical team member interface is shown in Figure 2. 

Figure 2. Metal Quality Team Interfaces 


COORS SUPPLIER 



The above matrix is an interface, not a hierarchy, and emphasizes membership on 
a team for a specific function. A team could have as few as two members, and as many 
as needed. Three or four is most common. The membership of a team varies with the 
nature of the task, and a team has no official life beyond the life of the task. 

For example, in incoming inspection of metal, the team begins with the material 
specialist and warehouse personnel who identify a rail car lot and sample size, and 
arrange to stage the sample to production lines. The team enlarges to include pro- 
duction operators, leadmen and metallurgical technicians only if the sample creates 
a problem. The team will identify the nature of the problem and isolate the shipment 
from which the sample was taken until the extent of the problem is determined. If 
the problem is not routine and requires metallurgical analysis, the team is joined by 
one or more of the staff metallurgists, who are regularly occupied by important, but 
less urgent R & D. 

If the problem is then recognized as going beyond just a coil or two, the sup- 
plier's sales representative is contacted, and possibly direct contact is made with 
mill metallurgists in the supplier's quality or manufacturing organization to begin 
the correllation of material performance and process data. The enlarging circle of 
team members could encompass the entire interface shown, and it has happened. The 
objective, however, is always to make disposition when the circle is the smallest. 

The emphasis in all problem solving is the solution and permanent corrective action, 
never the blame. This attitude has permitted us to develop mutual trust and share 
even proprietary company information, which we guard carefully. 

THE TECHNICAL PARTNERSHIP 

This close relationship has become what we call a "technical partnership" and 
is the sought-after relationship with all of our major suppliers. It is based on the 
simple recognition that the ideal quality position for both supplier and user is that 
which promotes the lowest cost to both. With this common objective, each is recep- 
tive to the efforts of the other to reduce cost or improve quality. In an industry 












where material cost can represent 60% or more of the finished product cost, your 
supplier's costs should concern you as much as your own. 

An example of mutual benefit derived from the technical partnership is a situa- 
tion in which one supplier approached us to request our use of a modified alloy for 
tab stock at a time when primary metal was in short supply and they needed to use 
more scrap. However, the average manganese level from using scrap would be higher 
than the tab alloy composition allowed, and would raise the strength level, with 
some loss of formability. The higher strength could be compensated by reducing the 
amount of magnesium, which offered some savings in alloy costs since the price of 
magnesium was three times that of aluminum at the time. 

Coors accepted the challenge, evaluated the modified alloy, which resulted in a 
further adjustment in composition and improvement in properties, and adopted it not 
only for our tab stock, but for our end stock as well. Many tab stock users still 
refuse to switch, while Coors enjoys annual savings exceeding $200,000, and know we 
have added to the latitude of our supplier's operation and enhanced the recycling of 
scrap into useful products. 

On another occasion, the same vendor came to Coors with a problem in which their 
remelt furnaces were being damaged as a result of the ignition of oil residues on re- 
cycled scrap from the can plant, and smoke emissions from the same source were un- 
acceptable to their local EPA. Very serious, indeed! While they continued to evalu- 
ate furnace modifications to handle the oil residues, Coors looked at our own system 
to see if the residues, then running about %%, could be reduced. An engineering 
project was established and implemented at a cost exceeding $100,000. In the mean- 
time, the supplier imposed penalties in its scrap buy-back policy that created a 
significant incentive, with different penalties at 4% and 2k% residual, and no pen- 
alty below 1%. 

By the time the penalties were in effect, Coors was below 2%%, and three months 
later were in the non-penalty level just below 1%. We continued our engineering plan 
and today ship scrap with about 0.5% oil residue, enjoying over $300,000 in reduced 
cost and taking pride in not contributing to the air pollution in southern Indiana 
and other places around the country. 

A third example of benefits of technical cooperation resulted from studies at 
Coors to correllate the tendency of certain lots of aluminum can stock to gall and 
build-up on tooling, creating a scratched can wall that was less shiny and was sub- 
ject to splits in the flange. Metal lographic examination of the metallurgical struc- 
ture was related first to the galling characteristic and then to composition. Work- 
ing closely with our suppliers, the correlation was verified, and the standard com- 
position of can stock now treats a former "impurity" as a necessary constituent with 
a minimum amount required. 

In still other programs we have significantly improved can stock quality, re- 
duced energy usage, improved metal recovery on remelt, evaluated and qualified 
cheaper alloys and new fabricating practices, all resulting in some incremental 
savings in fuel, metal weight in cans and better efficiency and higher profits for 
ourselves and our suppliers than might otherwise have been possible. 

The quality assurance of metals in can making at Coors contains most of the 
elements of quality systems everywhere, but its success is the result of highly 
motivated people participating, communicating and innovating through teamwork. 
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IMPROVING AUDIT EFFECTIVENESS 


WILLIAM J. HILL, CONSULTANT 
CONSOLIDATED EDISON CO., NEW YORK, N.Y. 10003 


INTRODUCTION 

During the past five years our audit teams have performed over 
150 audits. These audits evaluated: 

• The Construction Phase QA Program 

• The Operational QA Program 

• Compliance To Technical Specifications 

• The Qualification And Training of Personnel 

• The Corrective Action Cycle 

• The Emergency Plan 

• The Security Plan 

• Non-Radiological Environmental Monitoring 

Our analysis of these audits has identified five stages in the 
audit cycle where improvements can be made that not only make for 
a more efficient program but one that will instill more confidence in 
the results and obtain more meaningful corrective action. Most of 
these improvements can be instituted with a minimum of effort and will 
assure a more professional approach to auditing. 

ESTABLISHING THE PROGRAM 

There are four steps that should be taken prior to starting any 
audit program to assure that the program has a firm foundation. 

INDOCTRINATION - Senior management as well as management of the or- 
ganizations implementing the programs to be audited must know the 
objectives and limitations of the audit program. It is extremely im- 
portant that everyone in management realizes that an audit, no matter 
how well conducted, is no substitute for an effective QA Program (or 
any other program subject to audit). An audit can only measure and 
report on the effectiveness of a program at a particular point in 
time. It can not take the place of the program nor can it be used 
as an alibi for the lack of a program. 

ORGANIZATION - Because many different organizational elements imple- 
ment the programs that are subject to audit, the responsibility for 
audits is assigned to our corporate QA&R Department with an overview 
of the program residing in the Nuclear Facility Safety Committee 
(NFSC) . Furthermore, our audit group is totally independent of the 
line QA organizations responsible for developing departmental programs 
and procedures subject to our audit. If there were no corporate QA&R 
Department, our audit group could, report to the NFSC to assure that 
it had the organizational independence necessary for an effective 
audit program. This same independence should be provided in any com- 
pany in order to have a meaningful audit program. 

PRESENTATIONS - Senior management or the NFSC should be given periodic 
reviews of the status of the audit program to assure that the direction 
of the program continues to meet corporate policy and that organiza- 
tions responsible for taking corrective action do so in a timely 
manner. The mere fact that organizations know that the status of 
corrective actions will be reviewed by higher level management has 
been a definite help in obtaining timely action to remedy deficien- 
cies . 
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STAFFING - The audit team leaders are the key members of the audit 
group. They should meet ANSI requirements, as a minimum, and must 
be capable of training personnel to act as audit team members. There 
is no need to maintain a large audit staff, for the most effective 
utilization of personnel is obtained by having only the audit team 
leaders as permanent members of the audit group. Team members for in- 
dividual audits can be selected from other departments within the 
company (provided that they do not have any responsibility for the 
program being audited). 


PRE-AUDIT ACTIVITIES 

Prior to starting an audit, there are four steps that, if taken, 
can reduce the amount of time spent on an audit and/or improve the 
chances of finding non-compliances to regulatory requirements. 

PROCEDURE REVIEW - Most audit teams review those procedures pertaining 
to the subject undergoing an audit just prior or at the start of an 
actual audit. This is too late. The best time to review procedures 
is prior to their issue when they are circulated for comment by 
affected departments. If the comments of the audit group are obtained/ 
resolved at this stage, much wasted effort in training personnel on the 
procedure and implementing known deficient procedures can be avoided. 
Furthermore, if there is an appreciable time lag between the issuance 
of the procedure and a company audit to evaluate its implementation, 
this pre-release review may prevent the NRC from faulting it should 
they audit during this period. 

During any review of procedures, it is very important that procedures 
that cross departmental lines are coordinated with and agreed to by 
all affected departments. If they are not, the department that de- 
veloped the procedure may not obtain the cooperation of the other 
departments that are necessary in implementing the procedure, or worse 
yet, these other departments might not even know of the procedures 
existance. This can be very easily ascertained during the pre~issuance 
review performed by the audit group. 

A final check that should be performed during this pre-issuance review 
of procedures is to assure that all regulatory requirements have an 
implementing document that meets the full intent of the regulatory 
requirement. This is best accomplished by developing a matrix showing 
the relationship between the company's implementing procedures and 
regulatory requirements. Once this matrix is developed any gaps between 
requirements and procedures become quite obvious. Furthermore, if in- 
stead of gaps you have too many overlapping procedures, action can be 
taken to consolidate or eliminate them. 

FLOW CHARTS - Prior to performing an audit most team leaders prepare 
itemized check lists for evaluating the audit subject. A valuable 
supplement to an audit check list is a flow chart, especially for sub- 
jects, such as maintenance, procurement, etc. where the paperwork flow 
and decision process cross departmental lines. These flow charts are 
especially valuable in determining the "effectiveness" of a program 
and save auditors considerable time because key decision points in a 
process are obvious and by evaluating these key points a speedier 
assessment of the program is possible. 

SYSTEM TYPE AUDITS - Instead of auditing a single criteria of 10CFR50, 
Appendix B (e.g. Procurement Document Control) or a particular para- 
graph of the Technical Specification, we have found it much more ef- 
fective and less time consuming to audit complete systems. Our pro- 
curement cycle audit covers applicable portions of 13 of Appendix B's 
18 criteria so that we can evaluate the effectiveness of the procure- 
ment system in-toto rather than on a piecemeal basis. This not only 
saves the audit team's time but that of all personnel involved in the 
procurement cycle. It also increases our chance of detecting non- 
adherence to regulatory and company requirements and enables us to 
present one composite picture of the systems effectiveness to senior 
management. 
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ANALYSIS - Before starting an audit, team leaders review prior results 
for open items, reported corrective actions, or trend items that should 
be evaluated during an upcoming audit. These are not the only items 
that should be added to the audit's agenda. Previous NRC inspection 
reports, minutes of safety committee meetings, industry/ technical 
society meeting reports, trade journals, etc. all contain information 
that should be evaluated for application on your audits. The more in- 
formation that you use to guide your audits, the more effective your 
program becomes. 


CONDUCTING THE AUDIT 

There are four measures that can be taken during the conduct of 
the audit that can improve its effectiveness. 

TEAM SELECTION - Members of the audit team should have a thorough 
knowledge of the subject being audited and no direct responsibility for 
it. There is usually no problem in finding personnel to meet these 
criteria. In conducting audits of our QA program we have found it 
beneficial to use as team members personnel from the corporate QA staff. 
They developed the QA program and trained the operating and support per- 
sonnel in its use but have no direct responsibility for the operation 
of the program. Because of this they are ideally suited to evaluate 
the implementation of "their" procedures. Furthermore, by evaluating 
the implementation of the procedures they prepared, valuable insight 
is gained that is useful in revising procedures that have become out- 
dated or inefficient. Their presence is also helpful when reviewing 
deficiencies in implementing procedures at exit interviews. 

IDENTIFYING AUDITED MATERIAL - It is important to have a simple pro- 
cedure for identification. Identifying procedures reviewed during an 
audit so that they are not reviewed again on subsequent audits (pro- 
vided they haven't been revised) is relatively easy to do. However, 
identifying records that have been reviewed can be much more difficult. 
Limiting the scope of an audit to certain time periods is one method 
of preventing the same record from being reviewed more than once. 

Signing the record or stamping it is another. 

VERIFYING ACTIVITIES - The key activity on any audit is the verifica- 
tion that procedures are properly implemented. Aside from problems 
in scheduling to assure that the audit is performed when there is ac- 
tivity (e.g. as during an emergency drill), there are problems in 
determing which jobs, which personnel, which shift should be reviewed 
once the audit has started. Merely reviewing records of completed 
jobs does not give any real indication of whether or not governing pro- 
cedures were followed. A few general rules can be established that 
will increase the chances of detecting non-compliances to procedures, 

Namely: 

(1) Audit on-going jobs only. This enables you to verify that 
the material, tools and gages, work instructions, personnel 
qualifications are acceptable. 

(2) When auditing on-going jobs question the worker performing 
the activity. It doesn't do any good to just question his 
supervisor, for the worker is doing the job and if the 
worker either doesn't know what procedures to follow, or takes 
short cuts - there is a lack of control that must be corrected. 

(3) Check activity on second and third shifts. Make sure that 
information necessary to continue jobs properly is passed 
during the change of shifts. 

(4) When evaluating the procurement cycle, audit approved vendors 
to check on vendor approval procedures and visit vendors 
when surveillance inspection is being conducted to verify 
that the surveillance inspection plan is being followed. 


CONCURRENCE ON NON-COMPLIANCES - When a non-compliance to company 
procedures or regulatory requirements is detected, a management repre- 
sentative from the audited activity should be informed as soon as 
practical, preferably by bringing him to the area where it was found 
and reviewing the non-compliance with him. Ideally the audit report- 
ing forms should provide for his signature indicating that he has 
seen the non-compliance. This compels firm to start the corrective 
action process and eliminates unnecessary "hassling" during the exit 
interview. 


POST-AUDIT ACTIVITIES 

After completion of the audit, four steps should be taken to 
assist in the analysis of the results and implementation of corrective 
action. 

INDEXING - One of the major problems encountered in analyzing audit 
results is to determine if the trend shown by the results is getting 
better, or worse, or remaining essentially the same. Because of the 
different programs involved in auditing (QA, Training, Security, etc.) 
any analysis of results should be on a program basis. Because 
different audits may vary greatly in magnitude (i.e. in the number of 
inspection points observed) the "P" chart is the best one for an- 
alyzing the results. The sample size, n, associated with this chart 
can either be the number of checklist points evaluated or the number 
of records reviewed (especially appropriate for the corrective action 
audi t ) . 

SEVERITY LEVELS - Not all non-compliances are equally severe; even 
the NRC has recognized this. Therefore, when reporting your non- 
compliances, the method used by the NRC of having three levels of 
severity can be used. This does not make the indexing problem un- 
manageable as scaling factors can be used in conjunction with control 
charts or separate control charts can be established for the different 
levels of severity. 

FOLLOW-UP SYSTEMS - In order to assure that timely corrective action 
is taken on non-compliances found during audits, an efficient follow- 
up system must be maintained by the audit team and the organization 
that was audited. Both groups need follow-up systems, the audited or- 
ganization to assure action by them, the audit group to assure com- 
pletion and verification of the corrective action. To assure ef- 
fective follow-up our audit group publishes a monthly status on cor- 
rective actions and distributes copies to affected organizations. It 
is planned that our audit findings will become part of the company's 
Licensing and Regulatory Status Report which lists the status of 
all commitments to the various regulatory agencies. 

FEEDBACK - Audit results are normally transmitted to the audited or- 
ganization and senior management. Two additional organizations should 
be included on these reports - the training group and the technical 
group(s) responsible for generating procedures pertaining to the audit 
subject. The training group may decide, based on the audit results, 
to initiate or update training programs to indoctrinate or motivate 
personnel on company procedures. The technical group responsible 
for procedures may decide, based on audit results, to revise exist- 
ing procedures or initiate new ones in order to provide the control 
necessary to prevent future occurrences of non-compliances cited in 
the audit report. 


FUTURE PLANS 

The suggestions discussed above are ones we have reviewed and 
found helpful to our audit program. We are presently evaluating 
several items that we feel would be extremely useful to our audit 
program. 
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DATA BANK - All of the data generated by the Tech. Spec., Corrective 
Action, and QA Programs is presently prepared, analyzed, filed 
manually. If this data were entered on optically scannable mark 
sense records and cards both the generating station and the audit 
group could reap tremendous benefits. Continuously updated analyses 
could be performed, data retrieval would be very rapid, and the audit 
group could develop programs to selectively scan data and put key jobs 
on hold for audit. 

COST INFORMATION - There is a real need to develop accurate cost in- 
formation on your audit program. Not just the cost of your audit 
group but of the effort expended by the organizations undergoing audit. 
Our Cost Accounting System enables us to recapture the cost of our 
auditors on each discrete audit program but does not yet permit us to 
obtain cost information for organizations that we audit. This cost 
information, together with the previously mentioned data bank, will 
enable us to restructure our audit program for maximum cost effective- 
ness . 


CONCLUSION 

The methods presented for improving an audit program represent 
the results of five year's experience. Hopefully they can benefit 
your program as well as ours. As you gain experience and find better 
ways to run your audit program, share your knowledge as it benefits 
both utility and consumer. 
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IS YOUR QA ORGANIZATION AIMED AT THE RIGHT STAR? 


Leon V. Mason, Senior Engineer 
IBM General Products Division 
Boulder, Colorado 80302 


INTRODUCTION 

A functional staff management review provides the Quality Manager or a sub-unit 
of the Quality organization management with a peer evaluation of the overall effec- 
tiveness (doing the right things well) versus just efficiency (doing potentially the 
wrong thing without waste) of the organization. 

The top manager of the organization being reviewed establishes a two- to four-day 
agenda designed to cover the quality subsystems described later. Approximately 75% of 
the time is structured and the balance open to last-minute items of review-team 
interest . 


ISNESS 


Which Star In The Constellation? 

The basis for the entire review is the organization's mission (hopefully not 
impossible) which should be stated in a management-by-objectives format. The descrip- 
tion of the mission should define continuing responsibilities and then special one- 
year and three-year objectives. 

A useful way to look at the desired direction is t.o consider what things are 
designed to control change and then what objectives are meant to create change. 

If the review team is not familiar with the product menu, then you must provide 
them with a basic understanding of the products' purpose, function, and technologies 
as a necessary launching pad for their review. 

Strategies To Hook Your Wagon To A Star 

The details of a total quality system are covered much more eloquently by other 
papers at this conference and by quality control handbooks, such as the one edited by 
J. M. Juran, which provide checklists in excruciating detail. Since this review is 
effectiveness oriented, the best approach to understanding the organization strategy 
is to look at the operating philosophy in a minimum of the following quality system 
components : 

1. The procurement subsystem, or how does the supplier know what you want and how 
do you make sure that he gives it to you? 

2. The in-process subsystem, or does the operator have the tooling, measurement 
equipment, training, and data collection capability necessary to produce a 
quality product? 

3. The testing subsystem, or how well can you predict on a short-term and long-term 
basis what the customer will experience with your product? 

4. The new product introduction subsystem, or will the three subsystems previously 
mentioned have the proper resources at the right time to meet your new product 
schedule and cost commitments? 

5. The communication subsystem, or are there more feedback loops than in a modern, 
multi-level highway interchange so that the testers know what the customer 
thinks; the process people know what the testers think; the purchasing and 
suppliers know what the process experience is; and the new product planners 
understand how to better implement the next new product. 
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Another indicator of the strategy direction is the type of people within the 
Quality organization. An analysis for the review team of the organization's personnel 
by skills and/or job codes on one axis and degrees on the other axis gives one view of 
people placement. Another perspective is a second matrix showing product assignment 
on one axis versus prevention, detection, and failure (or resultant) manpower alloca- 
tion on the second axis. Another interesting view is shown by comparing skill level 
or product assignment by time in the job. 

BUSINESS 

Product Planning, Or Is Your Astrologer Up-To-Date? 

A small amount of preventive effort can be an indication that the organization is 
sacrificing tomorrow for today's crisis. TNT (The Next Thing) can truly explode next 
month or next year without adequate planning for new technologies or required new 
measurement techniques. 

The area of product planning lends itself to full documentation, which in turn 
allows rapid review. This review should concentrate on coverage of the entire product 
menu, the currency of the plan, the necessary detail, and the practicality of 
implementation . 

Quality Costs, Or Can You Afford To Get Off The Ground? 

Although it is tough in today's complex accounting systems to segregate the ele- 
ments of quality cost, the effort can be educational and revealing. The segments of 
quality cost typically are: 

1. Prevention Costs - The costs associated with planning, early analysis, informa- 
tion collection technology, and initial new product implementation. 

2. Appraisal Costs - The dollars spent on inspection or testing, regardless of 
whether it. 's done by an inspector or an operator, and the cost of the equipment 
required to do the inspection or testing. 

3. Resultant (sometimes called "failure") Costs - All the costs which would not 
occur if the total quality system produced no defects. These include reclamation 
activity, scrap, rework, field replacement, and customer complain* - handling. 

Distribution of the quality costs among the above categories will depend on where 
the product is in the development cycle, the complexity of the product and its simi- 
larity to previous products, the amount of new technologies used, and the philosophy 
of inspection. 

Product Liability Control 

The suggested approach here is to define a short, mock product-liability complaint 
to see how well the organization can respond to any consumer-product-safety reporting 
requirements, can track the product to the producing process, and can produce the key 
documents needed to verify commercially acceptable quality practices. 

WITHNESS 

Interfaces (If Not Love, At Least Respect) 

This phase of the review can be the most revealing and can guide the team in how 
to spend the 25% of unstructured time looking at questionable areas. The concept is 
to interview key management and/or working professionals in each of the areas which 
Quality services. Each area should be asked for an overview of the Quality organiza- 
tion's effectiveness. In addition, specific probing should include the following 
areas of interest. 

1. Purchasing 

Are standards and specifications clear? 

Is gauging compatible? 

Are peak workloads adequately planned? 

Is rapid and understandable feedback provided for in-process defects? 

Does the supplier rating system do its job? 





2. Manufacturing 

Are in-process quality goals defined? 

Are rapid measurement methods in place? 

Are critical process capabilities completed? 

Is there fast, understandable, problem feedback? 

Is there timely disposition of scrap and rework? 

3. Engineering 

Is process capability information available as required? 

Is an adequate design review in place to verify a measurement capability? 

Are problems fed back rapidly in Paredo order? 

Are special studies performed by Quality when requested? 

4. Sales (Marketing) And, If Applicable, Field Repair 

Is Quality responsive to special requests such as customer tours? 

Is there a timely, understandable, field problem feedback system? 

Does Quality understand the customer environment? 

Is Quality responsive to emergency field needs? 

Does any specific product's quality significantly hurt the sales effort? 
Professional Vitality 

A clue to troubles to come is a stagnant organization, not interested in renewing 
itself . 

Indicators of the vitality of the Quality organization include: 

Outside activities in professional societies and technical paper presentation. 

The amount of training undertaken - on the job, voluntary vocational, self- 
development, and university. 

The amount of job rotation. 

The number of patents and awards received. 

The number of significant innovations implemented within and/or by the Quality 
organization within the last year. 

Delegated Activities 

Under some Quality philosophies, inspection, measurement development, supplier 
interface, and similar activities can be delegated to organizations outside of the 
direct Quality one. The review team must carefully examine the two-way communications 
with the organizations to whom the activities are delegated and review the audit 
reports to make sure that this part of the total quality system is alive and well. 

The Quality manager must remember that even though everyone is responsible for 
quality, if he does not raise the flag on quality problems, his successor will. 

GETTING IT STARTED 


Timing 

This review is not designed for crisis situations and should not be held during 
peak seasonal workloads. The period from mid-November to mid-January should be avoided 
to maximize the benefit. One to two months' warning of the scheduled review should be 
given, and the final agenda, covering 75% of the scheduled review time, should be in 
place at least two weeks before the review. 

Selecting Participants 

Since much of the outcome will be subjective opinions, the reviewers must have 
the respect of those in the Quality organization, as well as of people in the areas to 
be interviewed. The team should be diverse, with some specialists covering such areas 
as process and testing, while others should have overview experience. Interest in and 
dedication to this type of activity is mandatory for each reviewer. Consider possible 
con r licts of interest between the reviewer and the Quality organization to make sure 
that the review does not turn into an axe-grinding exercise. 
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For small companies, the use of respected consultants would be useful. Perhaps a 
trade with other noncompetitive companies could be arranged. 

Motivation To Blast Off 

Why should any Quality manager or sub-unit manager in his right mind subject him- 
self to this kind of rigorous review? A flippant answer might be that, if he doesn't 
someone else will. 

A more likely answer is that any organization which is not currently in deep 
trouble can substantially benefit through new ideas and approaches from this type of 
review. Substantial benefit comes from the internal preparation for the review and 
the questioning that goes on at that time. The second major benefit is the suggestions 
and observations of the review team based on their years of experience. 

SUMMARY 

A Functional Staff Management Review (FSMR) can easily be misunderstood. Let's 
look at some of its "is" and "is not" characteristics. 

The Review IS: 

A peer evaluation of the entire quality system or subsystem. 

Results oriented. 

Based on professional judgement. 

Carefully planned in advance. 

The Review IS NOT: 

An audit that each procedure is being followed. 

Activity or product oriented. 

By the numbers. 

A surprise. 
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MANUFACTURING PROCESS OPTIMIZATION 
STUDIES --- THEORY AND USE 


Carl Mentch, Senior Statistician 
F. J. Borch Lighting Research Center 
General Electric Company 
Nela Park 

Cleveland, Ohio 44112 


INTRODUCTION 

Manufacturing operations seldom approach perfection even though there is constant 
pressure for improvement in terms of tighter tolerances and improved yields. These 
pressures may arise from high internal losses, from customer complaints, from cost re- 
duction programs, and from similar situations that bring out the fact that the process 
or equipment is not producing 100% good product. Sometimes the problems are obvious or 
easily discovered and corrective action may be taken. At other times the problems are 
not well defined or easily discovered and a "study" of the process is required. This 
paper defines, explains, and compares various studies that may be employed to achieve 
process optimization. 

Eugene Grant ^ has pointed out that, "Measured quality of manufactured product 
is always subject to a certain amount of variation as a result of chance. Some stable 
'system of chance causes' is inherent in any particular scheme of production and in- 
spection. Variation within this stable pattern is inevitable. The reasons for varia- 
tion outside this stable pattern may be discovered and corrected." The term "in 
statistical control" or more simply "in control" refers to variation within this stable 
pattern of chance causes. The term "out of control" refers to variation in excess of 
this stable pattern of chance causes, A stable system of chance causes is essential in 
determining process capability. 

Many people have used the term "Process Capability Study" to describe a wide 
range of different process studies, each of which is a constituant of what may be cal- 
led a process optimization study. In some cases the term "Process Capability" has been 
misused in the sense that a study based entirely on historical data, with no idea as to 
state of control of the process, may be a very poor indicator of the process capability. 
It is particularly important to be aware of the various elements of Process Optimiza- 
tion Studies when the conclusions reached from the study influence judgment as to the 
ability of the process to meet specifications or achieve a specified shrinkage rate. 

COMPONENTS OF A PROCESS OPTIMIZATION STUDY 

Considerable insight into the nature and intent of process optimization results 
from distinguishing four important components as constituants of a Process Optimization 
S tudy . 


A Process Performance Check is a quick check of the product produced by the 
process being studied. It is based on a small amount of data taken at a given point in 
time. It gives a quick picture of process performance within a limited time frame. 

A Process Performance Evaluation is a comprehensive evaluation of the product 
produced by the process being studied, usually based on whatever historical data is 
available. It gives a full picture of process performance over a period of time and 
can be used to estimate and predict potential process capability on the assumption that 
statistical control can be achieved. 

A Process Capability Study is a study undertaken to actually achieve a state of 
statistical control on a process. It is based on current data taken over a period of 
time and includes efforts to bring the process into control. Assurance that the true 
capability of the process has been achieved can be made only when control charts have 
been run on the process for a period of time. 

A Process Improvement Program is a comprehensive program undertaken to improve a 
process that is not capable of meeting specifications even though it is in statistical 
©1978 American Society for Quality Control 
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control. It includes engineering redesign of the process (possibly based on designed 
experiments) to improve the operation and the equipment so that the specifications can 
be met. 


IMPLEMENTATION OF VARIOUS PROCESS STUDIES 

Each of the elements described is implemented through the use of straightforward 
statistical techniques used to supplement and guide engineering know-how. They may be 
used separately or in combination to achieve process optimization. 

1 . Process Performance Check 


This is a short-time check of the product produced by the process being studied 
(utilizing usually a week or less of most recent data). In the case of variables data 
this check tells where the process is centered and what its variation was at the time 
of the check. A simple frequency distribution, or preferably a probability plot such 
as the frequency distribution analysis sheet (FDAS) is a good statistical tool to ana- 
lyze this check. With attributes data a knowledge of the overall percent defective and 
a breakdown of its components is usually all that is required. A Pareto analysis of 
the defects is a good approach for attributes problems. 

Although some people may call a study of this type a "Process Capability Study" 
it really doesn't tell much about the capability of the process. This is because with 
such data there is no way to know if the process was "in control" at the time of the 
study and there is no way to know if the observed variation (or defect breakdown in the 
case of attribute data) was typical of the normal process. An estimate of process 
capability based on a Process Performance Check is likely to be far more useful than a 
foreman's guess or "engineering judgment". However, if there is variation in the 
process over time or if the process is out of control at the time of the check, the 
estimate of process capability may be misleading. A Process Performance Check is usu- 
ally made by one person based on historical data. 

This, the simplest of the Process Optimization Studies, gives important informa- 
tion about the centering of the process and some information about the process spread; 
or for attributes data a knowledge of the percent defective and a breakdown of its com- 
ponents at the time of the check. It can lead to quick corrective action with resultant 
savings. However, most process problems are complicated and are not likely to be com- 
pletely solved by a Process Performance Check. Thus, while this check is useful and 
can lead to valuable corrective action, it alone is usually not enough to resolve most 
process problems. 

2 . Process Performance Evaluation 


This is a longer term evaluation of the product produced by the process being 
studied (usually at least one full month in duration). Such an evaluation shows how 
the process has been performing in the past. Usually there is too much data for prac- 
tical evaluation by a probability plot as in the Process Performance Check. Again, the 
Pareto analysis may be used to give valuable insight into attribute data. However, 
frequently the data is adaptable to control chart format (X and R for variables data 
and P or NP for attributes data). This format is usually best for a study of this type. 

Although a Process Performance Evaluation gives a broader basis to estimate pro- 
cess capability it is usually no more reliable than a Process Performance Check in that, 
without: special attention, it is unusual for a process to run "in control" for a period 
as long as a month. 

Sometimes it is necessary to estimate process capability from past data. If the 
data can be arranged in control chart form, a good estimate of true capability can 
frequently be made by eliminating the out of control points and estimating the process 
capability on the "in control" data only. 

Inasmuch as this estimate of process capability is based on the assumption that 
it is feasible to bring the process into control in a technical and economic sense, an 
assumption which is not always true, it is not advisable to use this estimate of process 
capability to set specifications. When it is necessary to set a specification to the 
process capability, it should be done on demonstrated performance in terms of consis- 
tently attainable levels based on a careful Process Capability Study as defined below 
and not on a collection of historical data adjusted to be "in control". In general, 
specifications should be set to the users needs not the producers capability. In the 
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rare instance where the user must accept the producers capability as his specification 
this capability should be demonstrated capability not projected capability. 

Being based on past data the Process Performance Evaluation is a fast yet thor- 
ough way to determine what has been going on (or wrong) in a process. It can lead to 
quick corrective action and thus to process improvement. While only a few process 
problems are simple enough to be resolved by a Process Performance Check many more 
processes may be optimized by the corrective action taken from a Process Performance 
Evaluation. 

A Process Performance Evaluation is a more valid way of calculating the capability 
of a process than the Process Performance Check but suffers from two major drawbacks. 

(1) It is based on what "has been done" not on what "can be done". 

(2) The assumption that it is practical to eliminate out of control 
points may not be true. 

A Process Performance Evaluation is usually (but not always) made by one person 
and is usually (but not always) based on historical data. 

3. Process Capability Study 

This is a study of current production, usually by means of control charts, whereby 
the true capability of the process is demonstrated. For processes involving variables 
data X and R control charts may be used to determine the process centering and variation 
that can be maintained on a continuing basis. For processes involving attribute data P 
or NP control charts are effective in demonstrating the true process capability. For 
both cases the process capability is determined when the process is running in a state 
of control. Achieving a state of control might well involve engineering trouble shoot- 
ing, establishing in-process controls, or tighter manufacturing supervision in the 
operation. It would not involve fundamental changes (improvements) in the process. 

Thus the "process capability" is the best performance that can be expected from the 
existing process running under normal "in control" conditions without any major expen- 
ditures for process improvements. A Process Capability Study, based on a process run- 
ning in control, usually shows that significantly better process capability can be 
achieved than that estimated from a Process Performance Check or a Process Performance 
Evaluation. Being based on current data, usually taken over the period of time in 
which the process is brought into control, the results are not available as rapidly as 
the results of studies based on past data. Occasionally a Process Capability Study may 
be terminated before full statistical control is achieved if: 

(1) The cause and effect of the out of control condition is known 
and minor and its correction is difficult or expensive. 

(2) The process meets the specification even with the known out 
of control condition. 

A Process Capability Study is frequently the most valuable of the Process Opti- 
mization Studies in that usually significant process improvements can be made at mini- 
mum expense. Every corrective action taken to achieve the true process capability also 
optimizes the process. In fact the process optimum is achieved only when the process 
is running at its true capability. While the Process Capability Study is usually in- 
expensive and results in yield improvement and cost savings, it requires skill and 
effort on the part of the study team. This skill and effort is directed at achieving 
an "in control" condition and developing process controls to maintain this condition. 

A Process Capability Study can be made by one person but is usually a team 
effort. The team consists of manufacturing and QC personnel and frequently includes 
engineering. 

4. Process Improvement Program 

Sometimes, even though a process may be known to be in control because of exist- 
ing in-process control charts or as a result of a Process Capability Study, the capa- 
bility of the process is inadequate to meet specification requirements. In this case 
manufacturing and engineering changes designed to improve the process must be under- 
taken. This may include major expenses for new equipment, better raw materials, im- 
proved flow of material, etc. These changes organized in a formal agenda showing items 
to be worked on, priorities, responsibilities, and a completion schedule, constitute a 
Process Improvement Program. A Pareto analysis of the problem areas will show where to 
direct the engineering effort. Process Improvement Studies should be undertaken only 
after Process Capability Estimates are well established. Statistically conceived and 
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designed experiments may be required to show what actions will be required to correct 
the problem areas. The techniques of evolutionary operations (EVOP) may be desirable 
to move the process to an improved level. The control charts from the Process Capa- 
bility Study should be continued to show the effect of changes made. A Process Capa- 
bility Study optimizes a process to the process capability. A Process Improvement 
Program is required only when this process capability is inadequate to meet the require- 
ments of the situation. The Process Improvement Program directs its efforts to improve- 
ing the process beyond its present capability to a new higher level. A Process Impro-ve- 
ment Program should be considered to be complete when the process is running at an 
acceptable rate, is "in control" and is producing product that meets the specif ications 
with a scrap rate so low that further expenditures on process improvement are not 
justifiable. In concluding a Capability Study or an Improvement Program, it is impor- 
tant that continuing controls be instituted to insure that the problems do not reappear. 

A Process Improvement Program is almost always a team effort. The team would 
usually include representatives from manufacturing, QC, and engineering. It might well 
be headed by a member of management. Such a program cannot be made from an engineer’s 
desk or from a distant laboratory. It must be made by interaction with the process 
itself. It takes time. However, when a Process Improvement Program is properly per- 
formed the end result is a process capable of running in control, to the desired 
specification, at an acceptable rate. 

SUMMARY- -COMPARISON OF COMPONENTS OF A PROCESS OPTIMIZATION STUDY 

The interplay of the four components of a Process Optimization Study may be 
summarized as follows. 

A Process Performance Check gives a "snapshot" of what the process is doing at a 
specific instant of time. It can be made rapidly from existing data or from data taken 
at the time of the check. It is useful to give quick information about the status of 
an operation after a change or when a problem is first identified. Such a check is 
usually more valid than a supervisors guess or "engineering judgment" but it does not 
include variation over time and includes no information concerning the state of control 
of the process. 

A Process Performance Evaluation gives a moving picture of process performance 
over a longer period of time. In fact this study gives the best information available 
about process performance when only past data is available. Since it includes more 
data, usually at least one month and sometimes all that is available, it takes longer 
to perform but gives a more complete picture of what has happened. If by chance the 
study was made during a period when the process ran in control, and this is not the 
usual case, the study would give a good idea of process capability. The Process Per- 
formance Evaluation should be used when it is important to know as much about the 
process as possible and it is impractical (usually because of time limitations) to do a 
Process Capability Study. 

A Process Capability Study gives a "live presentation" of what a process can do 
when it is running properly. It gives an indication of the ability of the process to 
meet the specification. Process Performance Checks and Evaluations can be made by an 
engineer at his desk using historical data. The Process Capability Study involves 
current data and includes interaction with the process in an effort to get the process 
in control. This is a "hands on" real time study and the study team must include 
someone with intimate knowledge of the process. The pay-off of this effort is in terms 
of better process stability and improved yields. (Minor expenses in terms of mainten- 
ance, training, and trouble shooting are normal. No major expenses are contemplated in 
the corrective action stage of a Process Capability Study.) A Process Capability Study 
takes longer to perform and requires more work than a Process Performance Evaluation, 
but it is much more useful in that it includes interaction with the process over time 
and shows what the process is capable of doing and how it reacts to changes. 

Ideally before a major process is turned over to production (by piloting or 
equipment development) a Process Capability Study should be run to demonstrate that the 
process and/or equipment is capable of: 

(1) Operating "in control" for sustained periods of time 
at an acceptable rate of speed. 

(2) Producing product within specifications. 


A Process Capability Study should be run on an existing operation when an important 
dimension cannot be properly controlled or when scrap is considered to be excessive. 
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The dollar savings due to increased yields will almost always far exceed the costs of 
engineering trouble shooting and supervisory attention required in the study. As 
processes or equipment age or as requirements become tighter, it would be well to per- 
form a Process Capability Study to determine the true capability of the process or 
equipment before making a capital outlay to redesign the process or order new equip- 
ment . 


A Process Improvement Program should be considered only when the composition of 
the picture must be altered. That is, when a Process Capability Study has shown that 
the process can run in a state of control but cannot meet the product specifications 
with an economic scrap rate. Since this study is long term and likely to be expensive, 
the first step should be a cost analysis to determine if the potential benefits warrant 
the expenditures necessary to achieve them. Such a study may involve substantial ex- 
penditures to improve the process. Designed experiments aimed at resolving the criti- 
cal process problems provide an excellent way to implement a Process Improvement 
Program. For some sophisticated processes evolutionary operation (EVOP) techniques or 
response surface procedures may prove to be the most productive method for implementing 
a Process Improvement Program. In cases where the required process improvements are 
obvious, the changes made should be monitored by a Process Performance Check after the 
corrective action has been taken. In any case, the control charts from the Process 
Capability Study should be continued throughout the Process Improvement Program. 

CONCLUSION 

When there is a need for process optimization and the corrective action is not 
immediately obvious, one or more of these studies should be considered. In a strategy 
for process optimization it is well to start at the earlier studies and proceed to the 
more complicated studies only as the need is demonstrated. A general rule, not to 
initiate a Process Improvement Program until a Process Capability Study has demonstra- 
ted the need for one, would frequently save money spent on process improvements when 
only process centering and adjustments are required. Proper use of these studies will 
identify the true capability of a process and effectively achieve process optimization. 
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TEMPERATURE MONITORING & CONTROL SYSTEM FOR MLB’S 


Robert J. Stubin 
Senior Manufacturing Engineer 
General Electric Company 

Aircraft Equipment Division/Aerospace Electronic Systems Department 
Utica, New York 13503 


INTRODUCTION 

The product thrust of the General Electric Aircraft Equipment Divi- 
sion, Aerospace Electronic Systems Department, is in the area of complex 
high-reliability electronic systems for aerospace applications at the 
lowest possible cost. Our customers are placing higher and higher speed 
requirements on digital circuitry while eliminating the availability of 
cooling air. Designs require custom digital LSI, semiconductors, and 
microprocessors. These trends necessitate complex high density elec- 
tronic packaging for our products. Therefore, the fabrication of com- 
plex multilayer boards represents a key manufacturing technology area. 
Typical designs average 14 to 16 layers and up to 4000 plated-through 
holes to provide electrical interconnections from one layer of circuitry 
to another. 

Hence, the multilayer board fabrication area was selected for the 
first microprocessor technology application in Manufacturing. This would 
provide some of the controls necessary to successfully manufacture com- 
plex multilayer boards with increased specification requirements at rea- 
sonable yield. A temperature monitoring system was built to demonstrate 
the feasibility of computerized monitoring and control of chemical proc- 
esses. The approach taken was to incorporate a microprocessor based mi- 
crocomputer used for manufacturing process control applications. 

INSTRUMENTATION AND CONTROLS 

The first step was to procure chemical indicators and controls that 
lend themselves to direct data transmission. Since the temperature of 
the plating tank must be maintained within close limits, initial effort 
was concentrated in this area. Also, temperatures can be easily read and 
the outputs can be transformed into binary coded decimal (BCD) outputs. 
Hence, procuring instrumentation with BCD outputs would simplify the ap- 
proach. It was determined that it would be advantageous to centralize 
indicators and controls where feasible. Fifteen chemical tank tempera- 
tures critical to the electroless and electroplating of printed wiring 
boards were selected. The chemical tank temperatures are monitored and 
controlled by digital indicators and controls located in two central con- 
soles (see Figure 1) . A microcomputer could then be directly wired to 
the central consoles for data collection and alarm reporting. 

DATA PROCESSING REQUIREMENTS 

It was understood that there are many chemical analyses which must 
be performed, but equipment is not in a state of development at this 
time in the entire field which lend themselves to direct data feedback. 
Therefore, it was assessed that any automatic data feedback system must 
eventually be married or interwoven with data taken from laboratory 
tests which are done manually. Figure 2 illustrates the overall system 
concept that evolved. 

Chemical tank temperatures are automatically interrogated every hour 
during chemical operations and are recorded for two weeks in the tempo- 
rary bubble memory storage existing in the terminal. Every two weeks or 
less this data is transmitted to a magnetic tape archival storage system 
resident on our Honeywell 66/60 regional host computer complex via a 1200 
baud direct communication link or 300 baud telephone hookup with modem. 
Manual data is inputted into the terminal keyboard as required. 

A Process Computer Systems Intel 8080 based microcomputer was se- 
lected for monitoring of the fifteen chemical tank temperatures. The mi- 
©1978 American Society for Quality Control 



274 


1978 ASQC TECHNICAL CONFERENCE TRANSACT IUNS — CnICAoU 



FIGURE 1. CHEMICAL MONITORING CONSOLES 



HOST COMPUTER COMPLEX REMOTE TERMINAL 


FIGURE 2. CHEMICAL MONITORING AND CONTROL SYSTEM (CMACS) 

crocomputer selected has the capability of monitoring at least 40 points 
and controlling 24 points (see Table I). In the future, these points may 
control/monitor additional temperatures, pH, fluid level, or ampere hours. 

A Texas Instruments (TI) 765 bubble memory data terminal was select- 
ed. It contains 20K nonvolatile storage, full ASCII keyboard, and a si- 
lent printer, with an acoustic coupler and RS-232 I/O ports. 









TABLE I. CMACS TEMPERATURE MONITORING 
TECHNICAL DATA 


8080-Based Microcomputer - Process Computer System, Inc. 

• 

IK Bytes RAM 

• 

IK - 7K Bytes PROM 

• 

Serial Port, 110 - 9600 baud 

• 

Software Programmable Time Base Generator 

• 

40 Digital Input Bits 

• 

24 Digital Output Bits 

• 

8 Level Interrupt 


MICROCOMPUTER TEMPERATURE MONITORING SYSTEM 
Signals from fifteen indicators are multiplexed to a common 8-bit 
data bus and are addressed on a 4-bit common bus as shown in Figure 3. 
The microcomputer is controlled by the TI-765 data terminal which prints 
out any alarm condition and a temperature monitoring file summary. 



FIGURE 3. MICROCOMPUTER MONITORING SYSTEM 


The temperature monitoring file consists of ASCII records. Records 
are separated by a blank line (two line feeds) . A record header includes 
hour, day, fiscal week, and year of recording. The actual record lists 
the indicator number and corresponding temperature in Celsius (see Table 
II) . Any temperature which is out of specification or out of preferred 
range is immediately printed out on an interrupt basis at the TI-765 
printer as a warning. Data which is temporarily stored in the bubble 
memory and permanently stored into the archival magtape system of the 
regional host computer can be readily recalled into the time share sys- 
tem for further data analysis and trend plotting. 


TEMPERATURE MONITORING TECHNICAL DATA 
The unique interfaces and multiplexers required to tie together the 
microcomputer, digital instrumentation and terminal printer are as fol- 
lows : 













TABLE II. CMACS TEMPERATURE MONITORING FILE FORMAT 


EXAMPLE : 




No. Characters 

24 

-53177 

<CR> 

<LF> 

10 

1 

75 

<CR> 

<LF> ' 


2 

74 

<CR> 

<LF> 

| 

3 

74 

<CR> 

<LF> 

[ 54 

9 

75 

<CR> 

<LF> „ 


10 

76 

<CR> 

<LF> ' 

j 


15 

76 

<LF> 

<CR> 

<LF> 1 

i-H Jr'' 

Jo 

— i 


• Two multiplexers (MUX) which switch data lines from 15 tem- 
perature indicators to the microcomputer external bus 

(TTL -> BUS) were designed and built. 

• An interface card which interfaces the external bus to the 
TTL-compatible microcomputer (BUS -> TTL) was designed and 
built. 

• An EIA to 20 mA converter which interfaces the microcomputer 
to the TI-765 terminal (1200 baud maximum transmission 
speed) was purchased. 

The microcomputer, and its associated interfaces, required for con- 
nection to selected digital instrumentation is illustrated in Figure 4. 



FIGURE 4. MICROCOMPUTER AND ASSOCIATED INTERFACES 


TEMPERATURE MONITORING SOFTWARE REQUIREMENTS 
Software routines were developed as follows: 

Microcomputer Operating System Software 

• IK Monitor - Provides editor/debug commands which may be en- 
tered from an ASCII keyboard. Commands include display mem- 
ory, fill memory, examine user stack register, and program 




execute. This 8080 IK monitor had been developed for use on 
the Process Computer Systems' microcomputer by the General 
Electric Special Purpose Computer Center for general company 
use and was obtained from them in the form of an even parity 
IK monitor source tape. An object tape was generated using 
an 8080 cross-assembler resident in the Regional Host Honey- 
well Level 66/60 time-share system. The object tape was 
edited and read into a PROM programmer to burn into a IK 
2708 PROM. Since this software activity had already been 
available, the software requirements for the project were 
significantly minimized. 

• Real Time Clock - An initialization program sets clock time 
and a timer subroutine generates an interrupt every 10 -2 
second. The real time clock subroutine (Timit program) was 
taken from the Intel User's Library manual which was avail- 
able in our Department. 

• User Program - A record list program prints out requested 
records ( example - see Table II) . 

Microcomputer Applications Software 

• Temperature Monitoring System Program - A sensor scanning 
subroutine, and an initialization subroutine for file gen- 
eration were developed. 

• Limit Alarm - A program to print out a warning when speci- 
fication or preferred range is exceeded and to extrapolate 
trend readings was developed. 

• Download Program - This program loads data from the bubble 
memory into the time-share system. 

Time-Share Software 

• File Interrogation Program - A program for record retrieving 
in archival storage from any time-share terminal was written. 


SUMMARY 

By utilizing the resources already available and by incorporating 
a standard microprocessor, manpower and technical expertise requirements 
to implement the project were significantly minimized. In addition to 
improved process control, this investment will result in a payback in 
three years. But, what is much more important, a body of knowledge has 
now been developed within the Manufacturing organization which will en- 
able expansion of microprocessor applications to other manufacturing 
areas for equipment and process control and monitoring where required. 

CONCLUSIONS 

The system as developed does provide real time process monitoring 
and control for selected chemical processes. 

• Continuous control and digital display are provided for 
chemical tank temperatures 

• A record is provided which shows that chemical processes 
are performed per specification 

• Product uniformity and quality are increased by maintaining 
the proper chemical solution environment 

• Operator awareness of chemical solutions is improved with 
the corresponding reduction in operating error, decreased 
rework, and increased yield. 

As additional sensors become available, the value of the system 
will increase. It is anticipated that the equipment suppliers to the 
chemical processing industry will start to produce digital instrumenta- 
tion and controls for a wide variety of measurements such as pH, speci- 
fic gravity, and ampere minute/hour. These instruments and controls can 



be readily connected to the existing microcomputer using the multiplexers 
and interfaces which have been developed. Also, all existing software 
modules can be utilized. 

With the addition of specific software subroutines for each major 
class of instruments, this application can be expanded to a totally in- 
tegrated real-time chemical monitoring and control system for most major 
chemical processing requirements. 
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Contingency Table Analysis of Continuous Data 


C. P. Veenstra 

Chevrolet Product Assurance 

L. C. Vance 

GM Research Laboratories 


G. F. Gruska 

Environmental Activities Staff 


General Motors Corporation 
Warren, MI 48090 


Within an industrial or service environment, it is often technically and/or economi- 
cally infeasible to implement the extensive production/ service testing of a char- 
acteristic using a specified measurement system. In these situations, an alternate 
measurement methodology is sought which is more compatible with the operating environ- 
ment. When both measurement systems are continuous, a traditional prerequisite for 
the acceptability of the alternate system is that (a function of) the systems be 
linearly related and highly correlated. Suitable limits (outpoints) for the alter- 
nate system can then be determined from the specifications using regression analysis. 

In its Notice of Proposed Rulemaking on Section 207(b) of the Clean Air Act, EPA has 
proposed a contingency table approach of defining correlation and establishing 
cutpoints in those situations where the alternate system is continuous, but does not 
satisfy the usual requirements. 

The following discusses the merits and applicability of this methodology with respect 
to good engineering and statistical practices. Examples are given based on the 
actual and simulated emission performance results of in-service vehicles. 


INTRODUCTION 


On May 25, 1977, the Environmental Protection Agency (EPA) published in the Federal 
Register a Notice of Proposed Rulemaking (NPRM) dealing with emission control system 
performance. In this NPRM, EPA asserts that the Clean Air Act requirements for short 
tests (ST) to implement Section 207(b) have been met and states that recently devel- 
oped EPA ". . .methods and procedures are in accordance with good engineering practice 
and are reasonably capable of being correlated with tests conducted under Section 
206(a) (1). . ." Section 206(a)(1) of the Act provides the authority under which 
emissions are measured according to the Federal Test Procedure (FTP). 

This situation EPA faced in developing this NPRM is similar to that often experienced 
by people involved in production quality control/audit. The goal is to find a 
measurement methodology suitable for production application which is an acceptable 
substitute for a more desirable yet costly and/or time-consuming test. 

When both measurement systems are continuous, as in the case of emissions testing, a 
traditional prerequisite for the acceptability of the alternate system is that (a 
function of) the systems be linearly related and highly correlated. This is ascer- 
tained by engineering analysis of the relationship assisted by an examination of a 
random sample of paired tests. 
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If a random sample of size n is drawn from a bivariate normal population with vari- 
ables X and Y, then the correlation coefficient r defined by 


r 


5Z ( x -t - x)(y. - y) 

1=1 

[toy- x ) 2 E( yi -y ) 2 ] 1/2 

i=l i=l 


and 


is a sample estimate of the population parameter p = 


Cov (x,y) 

a a 

x y 


The coefficient r is sometimes referred to as the sample product-moment correlation 
coefficient. This is one of the most commonly used measures of correlation. 

However, EPA states "(i)t was anticipated that the short tests would not correlate 
with the FTP in a classical statistical sense, i.e., no short test is capable of a 
reliable and consistent prediction of the FTP mass emissions." 

The proposed EPA rule further states that ". . .the Clean Air Act requirement of 
reasonable correlation. . . (will) be met if the short test is capable of reliably and 
consistently predicting whether the vehicle would pass or fail the FTP even if it 
cannot give the magnitude of the passing or failing margin." 

Under this rationale, the continuous data should be dichotomized into pass (P) or 
fail (F) classes with respect to each of the tests. This will then yield a 2x2 
contingency table as follows: 


Table 1 


Federal 

Test 

Procedure 

(FTP) 


Pass Fail 


PF = E 

o 

FF 

PP 

FP = E 

c 


ST 

Cutpoint 
Short Test (ST) 


Fail 


Pass 


where 

PF = pass ST & fail FTP = E £ 
FP = fail ST & pass FTP = E 


error of omission 
error of commission 


CONTINGENCY TABLE CORRELATION 


The 207(b) Support Document favors the use. of a contingency table approach for deter- 
mining correlation between the FTP and the short test because given a short test 
limit (cutpoint), the contingency table method can satisfy EPA’s interpretation of 
the Clean Air Act requirement of establishing correlation between the FTP and one or 
more short tests, i.e., it meets the requirement of predicting FTP pass-fail. A 
measure of contingency table correlation is the phi coefficient. 

(PP) (FF) - (E )(E ) 

/ _ £ 2 

<t> - i/2 

[ (PP + E ) (PP + E ) (FF + E o ) (FF + E c > ] ' 




It can be shown that if the bivariate normal variables x, y are dichotomized, then </> 
is p. 


The phi coefficient has a number of serious deficiencies, however. Carroll (1961) 
has shown that if either or both characteristics are dichotomized by cutting a con- 
tinuous distribution into two parts, then the value of <t> depends strongly on where 
the cutting point is set (see also Fleiss and figure 6). Thus, the correlation 
coefficient can be increased or decreased with arbitrary choice of outpoint. For 
example, given 61 observations distributed in the following manner: 

60 points in PP quadrant, 

1 point in FF quadrant, 

0 points in E^ quadrant and 
0 points in E q quadrant. 


the correlation coefficient is equal to 1.0. However, if the short testcut point is 
increased so the vehicle in the FF quadrant falls in the E q quadrant (i.e., passes 

the short test, fails the FTP), then the correlation coefficient is undefined (0/0). 

Thus, by looking only at the correlation coefficient, in the first situation, the 
short test may be interpreted as an ’’excellent" predictor of FTP pass/fail but in the 
second situation, the short test is "horrible." 

Although the observed contingency table correlation coefficient {5 is the maximum 
likelihood estimate of (6 under multinomial sampling, a further problem arises in 
determining the reliability of the sample estimate of correlation coefficient if one 
accepts the contingency table approach and uses such a coefficient as a measure of 
correlation between the FTP and the short test. 


The approximate large sample variance of j> is 

« 1/n [1 - b 1 + («* + 1/2 <6 3 )*A - 3/4 <£ 2 


where 

(R 

A = 


3L r 2 )(c 1 - c 2 ) 


(c 1 c 2 r 1 r 2 ) 

( R 1 - R 2 ) : 
! -r^ 


1/2 


(C 1 " V 
C 1 C 2 


(B + D) ] 


= E + FF 
o 

= E + PP 
c 


= E + PP 
o 

= E + FF 
c 


To estimate the asymptotic variance from sample data, we substitute ? for </>, and the 
observed marginal proportions for the population values. Because is asymptotically 

2 

normal with mean t and variance a ^ , we can use ? and the estimated to construct 
confidence intervals for <k>. 


In order to avoid ambiguity, a specific set of data has been selected for illustra- 
tion purposes. Data on GM vehicles from the 1976 EPA Restorative Maintenance Program 
will be used. The description of this data is: 

Number of vehicles tested: 102 (all vehicles test "as received") 


Emission selected 
Short test selected 
Units on HC for FTP 
Units on HC for ST 


HC 

Federal 3 Mode - 30 mph 

gm/mi 

ppm 



This HC data has the following distribution: 


81 points in pass-pass (PP) quadrant, 

3 points in fail-fail (FF) quadrant, 

4 points in error of commission (E ) quadrant and, 
14 points in error of omission (E q ) quadrant. 


With this quadrant distribution, the estimate of the contingency table correlation 
coefficient was 0.19. At the 90% confidence level, an approximate interval estimate 
of the coefficient was from -0.02 to 0.41. 

The maximum value of may be a value actually less than unity. Qualitatively, the 
more different the row and column marginals, the lower the upper limit on the absolute 
value of <t>. Specifically, if the absolute value of were related to the error of 
commission and the error of omission E q , it can be shown analytically that for 

fixed E c and E q rates, <b is maximized only when the PP and FF proportions are equal. 

An example of the sensitivity of the maximum values of </> are to E and E are given 

in Table 2 for fixed E = .05. 

c 

Table 2 




E 

c 

= .05 



E 

o 

Max $> 

E 

o 

Max (4 

E 

o 

Max 

.00 

.905 

.11 

.686 

.21 

.519 

.01 

.883 

.12 

.668 

.22 

.503 

.02 

.862 

.13 

.651 

.23 

.488 

.03 

.841 

.14 

.633 

.24 

.473 

.04 

.820 

.15 

.616 

.25 

.458 

.05 

.800 

.16 

.599 

.26 

.444 

.06 

. 780 

.17 

.583 

.27 

.429 

.07 

.761 

.18 

.566 

.28 

.415 

.08 

.742 

.19 

.550 

.29 

.401 

.09 

.723 

.20 

.535 

.30 

.387 

.10 

.704 






OUTPOINT METHODOLOGY 
Besides EPA also discusses the following measures. 


(a) 

commission error rate 

E 

c 

E + FF 

(c) 

FF 

test effectiveness , 

E + ft 



c 


o 


E 

+ FF 



(b) 

test rejection rate 

0 

+ FF 

(d) 

error of commission E 

c 


The following limit guidelines were also given for "acceptable correlation: 


E 

c 


(a) 

c 

E + FF 
c 

< .15 

, x ff 

^ E + FF 

> .5 


E 


o 


(b) 

c + FF 

E + FF 
o 

< 1.0 

W B c <£ 

.05 


A cutpoint would then be selected for a short test by an iterative procedure until 
one or more of the above were met. 

A Monte Carlo Simulation study was conducted to determine the prediction ability of 
the different cutpoint methods. In particular, the study compared the prediction 
abilities of the error of commission set to 5% method to the short test rejection 

ratio set to 1.0 (E = E ) method. For both methods, the short test and standard 
c o 



tests were taken to have the same average and scale parameters. In effect, the 
standard test is being compared to itself by simulating back-to-back tests. This was 
done since the standard test itself represents the best possible (i.e., correlatable) 
"short test" and the effectiveness of the method can readily be determined. 

The two tests are assumed to be log-normally distributed with a skewness of 1.2 and a 
coefficient of variation of about 40%. Different degrees of correlation (0, .5, .9) 
were investigated along with different levels of percent nonconformance (10%, 25%, 
40%) on the standard test. To standardize the results to a general case, it is 
assumed that the limit value of the standard test above which all tests fail is 1.00. 
Although this simulation was designed to emulate the standard emissions test (FTP), 
the results are readily generalizable to any situation. 

RESULTS 


Results of the simulation for a correlation coefficient of 0.5 are shown in Figure 1. 

ERROR OF COMMISSION SET TO 5% (E c »5%) METHOD - Based on the model assumptions, the 

theoretical short test cutpoint should be 1.00 and any variation from 1.00 is a 
measure of the error induced by the random sample distribution of the 50 selected 
paired data points. 

Cutpoint Variation - For a given correlation, as the percent over the standard 
test limit increases, so does the value of the cutpoint. For a given percent 
over the standard test limit, as the correlation increases, the average short 
test cutpoint decreases. For the E =5% method and a degree of correlation of 

0.5, the cutpoint ranged from 0.87 to 1.57 or 70% of theoretical correct value. 
This is quite- substantial when one considers that the proposed methods would 
apply to all groups of vehicles regardless of the amount of correlation and 
percent over the limit. The highest cutpoint obtained was 1.98 with correlation 
of 0.0 and 40% over the FTP limit and the lowest cutpoint was 0.82 with correla- 
tion of 0.9 and 10% over the FTP limit. This is the total range of 1.16 or 
116% of the theoretically correct value. 

Cutpoint Coefficient of Variation - A measure of the variability of the short 
test cutpoint is the coefficient of variation. For this simulation, it varied 
from a minimum of 2.9 to a maximum of 9.0. 


Error of Omission Variation - The range on the average error of omission in the 
sample was from 2% to 36%. 

Variation in Fraction of Correct Predictions - The lowest average correct pre- 
diction is 0.59 and the highest is 0.93. For a correlation of 0.5 and a FTP 
nonconformance rate of 25% (25% over the FTP limit), the average is 0.77. 

SHORT TEST REJECTION RATE SET TO 1.0 

Cutpoint Variation - As shown in Figure 1, the average value of the cutpoint 
remains nearly constant at about 1.00 (the theoretical value) for all levels of 
correlation and percent defective, but the range of the cutpoint was from 0.81 
to 1.59. This is a total range of 0.78 which is somewhat less than that for the 

E =5% method; however, with such variation, it still remains possible that all 
c 

group populations cannot be treated equally. 

Cutpoint Coefficient of Variation - The coefficient of variation for the cut- 
point varies from a minimum of 2.4% to a maximum of 12.1%. 

Error of Commission (or Omission) Variation - The range of the average error of 
commission (of omission) was from 3% to 24%. 

Variation in Fraction of Correct Predictions - The lowest average correct pre- 
diction is 0.53 and the highest is 0.94. 

Comparison of Cutpoint Methods - The rejection ratio set to 1.0 method consistently 
gives an average cutpoint close to the theoretical value of 1.00 whereas the error of 
commission set. to 5% gives an average cutpoint substantially higher than 1.00 except 



when the correlation is very high. To understand these trends in the cutpoint estimate, 
it is necessary to look at the integration surfaces. The diagrams in Figure 2 represent 
the boundaries of the integration surfaces. 


Figure 2 
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1 .0 


Short Test 
Val ue 


Short Test 
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With more correlation, the boundary becomes more elliptical. With a higher percent 
over the standard test limit, the percent of the short test distribution greater than 
1.0 becomes larger. 

For the error of commission set to 5% method, it can be seen from the diagram that a 
correlation of 0.0 and a percent over the standard test limit of 10%, the cutpoint 
will be greater than 1.0. With a percent over the standard test limit of 40%, the 
cutpoint will be larger than that with a 10% over the standard test limit. With a 
higher correlation, the cutpoint will decrease in value but still be larger than 
1.00. Values less than 1.0 are obtained in the simulation due to the random samples 
being generated. 

On the other hand, with the short test rejection ratio set to 1.0 method, the cutpoint 
values will oscillate about 1.0 due to the mirror symmetry of the boundary with 
respect to the 45° line through the surface. 

Another significant difference between the two methods lies in the error of commission. 
While for the E c =5% method, the error of commission was set to 5%, in the E c =E q 

method, the error of commission averaged as high as 24%. This occurred with correla- 
tion to 0.0 and 40% over the FTP limit. Such high errors of commission are undesir- 
able. On the other hand, if a high correlation (p) exists, such as 0.9, then the 
average error of commission will be in the neighborhood of 3-6% of the E c =E q method, 
depending on the percent over the FTP limit. 



Conclusion 


The above discussion shows that for E c =5%, and E c =E q , the proposed methodology is 

highly erratic and dependent on the population characteristics. Figures 3 through 6 
indicate the same is true for the other proposed measures. In general, only if the 
short test is highly correlated in the classical sense with standard test is the 
methodology acceptable; but if this is the case, the question naturally arises, why 
not use "traditional" regression techniques? 
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A SYSTEMS APPROACH FOR DEVELOPING SUPPLIERS 


Leo R. Harrington, Quality Control Manager 
Chrysler Corporation, Introl Division 
Ann Arbor, Michigan 48106 


Effectively controlling the quality of purchased material in automotive compo- 
nents manufacturing has always been a prime challenge to Quality Control professionals. 
No matter what technology they employ it seems that the results is always less than 
totally satisfactory. 

Common among the systems tried are: 

(1) Receiving Inspection using statistical sampling methods and programed instruc- 
tions . 

(2) Supplier certification. 

(3) Onsite (at the supplier) acceptance inspection. 

The objectives of controlling quality of supplier material is well known. The 
principal objective is to provide assurance that materials we will input to the manu- 
facturing process are functionally satisfactory. This statement translates into mate- 
rial which will not require off standard labor to use, create scrap, or contribute to 
premature field failures. Naturally, the buyer wants the value he contracted for, and 
for which he is paying the contract price. 

Today when we setup a new manufacturing line, great care is taken to maximize pro- 
ductivity, control* costs , control inventory, balance operations, control quality, and 
to provide a proper environment for the worker. This new line becomes an intergrated 
system with the foregoing listed elements, all interfaced and interacting. Great care 
is taken to program the quality control operations and to provide adequate test equip- 
ment. In-process test points are specifically identified. Final lot control inspec- 
tions are structured. 

The objective of the quality control can be described as being able to know from 
line supplied data what quality of product we have produced during the shift. If we 
truly know this quality, then we can say we know the quality of the material we ship 
and really meaningful quality control has been accomplished. There must be the capa- 
bility to contain substandard quality product and to correct quality failures before 
the product is returned to the process or accepted as finished product. 

The establishment of effective quality control in-process tests is made for any 
in-house fabrication. 

When it comes to the material to be obtained from outside suppliers the intense 
programming of quality control technology in the supplier's operations is usually not 
undertaken. In other words, we expend most effort in-house to develop an effective 
quality control system, but seldom do we get that involved with the supplier. The most 
we do is process purchased material through a receiving inspection. 

A SYSTEM APPROACH IS NECESSARY 

A manufacturing system has an enviroment parameter of the facility that houses it. 
The system module has inputs of materials, labor, energy, and information. These in- 
puts enter the assembly or manufacturing process where they are transformed. From the 
transformation process come outputs of finished product. From the stream of outputs 
flow comes feedback concerning the output to provide process control. Any supplier of 
part or component to the manufacturing system is providing an input to the production 
module . 

The supplier in reality is operating a subsystem module in the manufacture of his 
part or component. This subsystem module is an input/output transformation process. 
When the output product becomes an input to the buying company's system, the two mod- 
ules are interfaced and are interacting. Such is the case for every supplier to the 
major system. 
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The supplier in reality is operating a subsystem module in the manufacture of his 
part or component. This subsystem module is an input/output transformation process. 

When the output product becomes an input to the buying company's system, the two mod- 
ules are interfaced and are interacting. Such is the case for every supplier to the 
major system. 

This situation should indicate that the prime manufacture should view the quality 
control applications in a total system module context. Further, he should adopt strat- 
egies to make his quality control applications a total system approach. 

THE SUPPLIER MUST KNOW THE QUALITY HE IS SHIPPING 

It is a known fact that suppliers who have the means of knowing in terms of signi- 
ficant data what quality they ship seldom cause the buyer a quality problem, and when 
they do, it usually is traceable to a design limitation. Such is not always the case. 
Small manufacturing operations may be planned to produce a part through a sequence of 
line operations completely devoid of any checking to confirm compliance to specifica- 
tion. Any feedback which would cause correction of manufacturing errors should origin- 
ate with the buyer's operations. A sensing point this remote from the problem initiat- 
ing point is of little value as an effective quality control feedback. 

THE SYSTEM SURVEY 

The system approach for developing suppliers requires that qualified Quality Con- 
trol Engineers perform an audit survey of each contemplated supplier of material. The 
objective of this survey is to acquire knowledge of a source's quality control capa- 
bility. 

Information sought would include: 

(1) Management attitude towards Quality Control technology. 

(2) Management attitude towards making basic quality control costs expenditures. 

(3) Is there a Quality Control system organized and operational? Describe. 

(4) Company's organization structure. 

(5) Size of* the company. 

(6) Type of products produced. 

(7) What technical expertise do they have? 

(8) Is there a problem of proprietary concern? How extensive? 

The audit would provide means to pinpoint quality control deficiencies. The main 
concern is the supplier's ability to derive from his quality control system knowledge 
of the quality of the material he is producing, and the action he takes on nonfunc- 
tional quality when it is produced. When a survey reveals this type of deficiency 
existent in the system the survey report communicated the fact to the Purchasing de- 
partment. In fact, each audit survey results in a report to Purchasing. Such reports 
will: 

(1) Approve the source as quality control approved. 

(2) Withhold approval until certain cited conditions are corrected. 

(3) Rejects the source as a supplier. 

QUALITY CONTROL INTERFACE WITH PURCHASING 

The closest coordination between Quality Control and Purchasing is required to 
initiate the system approach of improving supplier performance. The application of 
quality control technologies to supplier operations must come about because of Pur- 
chasing desire to have improve supplier performance. Purchasing breaks the ice with 
the supplier, initiating the request to the supplier for audit surveys, operating 
system changes, and makes disposition of all cost claims. Quality Control involve- 
ment is one of a technical expert assisting Purchasing in resolving system ineffi- 
ciencies and deficiencies. Quality control must acquire expertise in handling sup- 



plier relations so that Purchasing can have confidence in Quality Control. The repre- 
sentative working with the supplier’s personnel without Purchasing personnel in atten- 
dance. Once such confidence is developed, Purchasing can derive many services from 
Quality Control. One such service is the onsite expediting of samples. Another is 
the conducting of educational seminars at the supplier's plant. Let it be emphasized, 
that there must be domination in all Supplier/Purchasing contacts by Purchasing for 
the system approach to improving suppliers to be effective. The strategy calls for 
doing a better job of developing suppliers and to expand the content of a company's 
quality control program. 

INITIAL INSPECTION SAMPLE REPORTS (I.S.I.R.) 

One of tne key elements of the system approach to improving supplier performance 
is the requirement that the supplier furnish an initial sample. Such a sample pro- 
duced from the production process demonstrates the process capability to produce the 
product. The requirement made of the supplier that he measure the sample as to its 
compliance to all engineering specifications, perform all performance tests, and to 
document the results, forces the supplier to become totally involved in his contract 
commitment. Disposition of the results is made in joint session with supplier repre- 
sentatives and buyer representatives at the supplier's location. An I.S.I.R. report 
accepted by the Purchaser’s Quality Control is necessary for payment of the supplier's 
tool bill. The latter provides an incentive for the supplier to produce a good qual- 
ity sample and to correct process problem at an early date. 

The I.S.I.R. requirement forces the supplier to become acquainted with all speci- 
fications at an early time in his contract. It, also, forces the procurement of mea- 
surement capability in the supplier facility and training of a person to perform the 
measurement action. Such investment puts strength in the supplier's long term com- 
mitment to a meaningful quality control operation. 

Frequently, processing of the I.S.I.R. will reveal non-compliance to the specifi- 
cations. The purchaser's quality control representative can then assist the supplier 
in obtaining special disposition of such problem with the purchaser's engineering 
activity. Temporary deviations or corrective rework programs can be worked out to 
allow time for permanent correction and to supply parts to meet immediate requirements. 
This type of assistance to the supplier does much to establish a firm rapport between 
the respective quality control activities. 

QUALITY PROBLEM REFERRAL 

After the suppliers and purchasers quality control programs are fully operational 
a quality problem may occur in the purchaser's plant. When this happens, the problem 
is referred to the quality control activity. After investigation and identification 
of the problem, Quality Control will contact the supplier. The first order of busi- 
ness is to determine if a failure occurred in the supplier's quality control system. 

Answers to certain questions are pertinent to taking corrective action. Is the 
incident a person failure or a process failure? Did the system sensing characteris- 
tics detect the failure? Why was the effected product not contained in the supplier's 
plant? What specific action needs to be taken to prevent reoccurence? What action 
needs to be taken with the defective inventory? 

The corrective action is concentrated on system failure correction when a quality 
problem is defined. The correction of defective inventory does not overwhelm all 
other concerns. 


QUALITY ENGINEERING REQUIREMENTS 

The quality engineering staff required to successfully implement a total quality 
control system with supplier must be comprised of high calibre personnel. Engineers 
should have education cores in technical engineering sciences. Further education in 
business management is preferred. Experience in Purchasing or Sales adds to the man's 
value. The engineer must like working with people and understand the problems and 
personality of the entrepreneur. Each engineer needs to have three years training in 
applying quality control technologies. 

RECEIVING INSPECTION 

A good functioning quality control program interfaced with the supplier's plants 
quality control will make receiving inspection a needless activity. The money saved 
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by not operating a receiving inspection activity can be used to pay for the operation 
of a quality engineering staff to be devoted to supplier services. Usually there will 
be a favorable savings even after the cost of quality engineering services are sub- 
tracted . 


BENEFITS 

Once the program is fully operational, certain programmed in-process performance 
tests become redundant and no longer necessai-y. Then the manufacturing process can be 
realigned. The cost savings can be significant. 

An example of this, a major component supplied out of an industry with numerous 
competitors. Competitive bidding to obtain contracts had reached a point that sig- 
nificant quality control procedures were either eliminated or downgraded to a point 
of non-effectiveness in order to meet competitive pricing pressure. Consequently, the 
industry was shipping a significant amount of bad product. The purchaser was inputting 
such material into his manufacturing system and order to protect his build quality 
had to 100% test his product. Reject levels reached the forty-five percent level; 
repair cost escalated. In conjunction with Purchasing, and with the supplier's coop- 
eration, a system analysis was made. Strategies were developed to correct the suppli- 
ers quality problem. The revision of the supplier's manufacturing system caused pro- 
cessing changes to take place where the cost factors offset or balanced out. The 
forty-five percent reject level dropped to five percent. At the new level of defec- 
tiveness the one-hundred percent level of product testing was no longer necessary and 
could be eliminated. Because the volume of production was high, extremely high saving 
could be realized. 


SUMMARY 

The system approach to improving supplier performance can be effective. To in- 
stall and implement the program is time consuming. An added burden placed on Quality 
Control and on Purchasing. Improved supplier performance and better quality are the 
rewards for the hard work involved. The Quality Control professional should be pleased 
at the opportunities to advance the use of quality control technologies in the small 
business firm. 

This approach to the control of purchase material quality may be the only way we 
can reach the entreprenaur and convince him that quality control should command his 
attention in planning his business operations. 
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RESULTS ORIENTED FOUNDRY QUALITY ASSURANCE 


S. C. Cochran, Manager, Product Quality 
International Harvester Company 
5565 Brookville Road 
Indianapolis, Indiana 46219 


For most companies, quality is a vital concern and an integral 
part of the overall production picture. At the Indianapolis Foundry 
of International Harvester, management has constantly sought to improve 
the quality of its castings, and to stress the importance of quality to 
workers . 

A program with those goals was started in 1974, and its results 
have been encouraging. Before the program could be developed and im- 
plemented, however; management had to take an in depth look at available 
equipment, manpower, and the quality group structure. 

In some ways, the Foundry has not changed since the 1974 program 
was initiated. The Indianapolis facility is strictly a gray iron 
foundry utilizing electric induction melting with six 25- ton Brown- 
Boveri furnaces capable of producing 900 tons per day. Of the 
foundry's 400,000 square feet of floor space 110,000 square feet are 
occupied by core making activities. In the molding area are these four 
major Molding Units: 

I. Hydra-Slinger 38" x 51" - 16" over 12" at 75 molds 

per hour. The S linger produces engine blocks, cylinder 
heads and housings. 

II. Spo Automatic High Pressure Squeeze 24" x 32" - 9" 
over 9" at 220 molds per hour. This unit makes 
cylinder heads, small brake drums and flywheels. 

III. Osborn Automatic High Pressure Squeeze 24" x 32" - 
10" over 12" at 300 molds per hour. Intermediate 
size castings such as intake and exhaust manifolds, 
water pumps, oil pumps and flywheels come from this 
unit . 

IV. Spo Automatic Squeeze 51" x 38" - 12" over 14" at 
200 molds per hour. The Spo Automatic Squeeze 
produces such castings as large brake drums, housings, 
and cylinder heads . 

One event in 1974 that, to some extent, inspired the quality im- 
provement program was the separation of the Foundry from the Machine 
Shop of the Indianapolis Plant. With the introduction of the 
Manageable Unit Concept at the Plant, the two areas, which previously 
had been managed as one entity, were divided into two relatively 
distinct sections. At that time, the foundry quality group was re- 
structured and became directly responsible to the Foundry Manager. 

Where there had previously been six management, six salaried, and 
30 hourly employes, there were now 10 management, 18 salaried and 30 
hourly workers, all of whom were utilized in a different manner. 
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With the increase in personnel and the shift in responsibilities, 
new job assignments were necessary. The salaried group had to be 
structured to provide services for the production departments in day- 
to-day operations. They would also need to be utilized for documen- 
tation of procedures, performance of audits to identify discrepancies, 
development of records of performance, and statistical analysis work. 

A careful adjustment of personnel and assignment of duties resulted in 
this line up: 



I. One Process Control Analyst and one Quality Control 
Clerk to develop documents of procedure for all 
operations, from core making and cleaning to re- 
fractory coating and assembly, and including molding, 
pouring process times and temperatures , and any other 
procedures needed to produce a uniform, acceptable 
quality casting at minimum cost. 

II. Four Quality Control Inspectors (two per shift) on 
molding units, reading and recording iron temperatures, 
pouring times and mold hardness. They also were to 
sample metal superheat, and spot check cores and core 
assemblies at the unit. 
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III. Four Quality Control Inspectors (two per shift) in 
the Core Room, auditing processes at the making, 
cleaning, refractory coating, and assembly of cores. 
Recording oven temperatures, measuring specific gravity, 
checking contamination of refractory coatings and 
miscellaneous checks as needed were also written into 
their job description. 

IV. Two Quality Control Investigators (one per shift) 
checking and following samples, auditing specific 
trouble jobs and investigating identified problems. 

V. One Quality Control Investigator who would serve as 
Machine Shop liaison. 

VI. One Statistical Analyst to keep records, reports, etc. 

VII. Three Metallurgical Observers to develop material 
standards and audit incoming material for conformance 
to specifications. These people were also made 
responsible for auditing the melt process and pro- 
cedures, submitting reports and keeping records. 



VIII. One Steno-Secretary to serve the entire quality 

group as the need arose for communications, letters, 
reports and charts . 

As with most changes, these increases had to be justified finan- 
cially, using a method of evaluation new to International Harvester. 
The Corporate Quality Assurance group developed a systematic approach 
of measuring quality costs for internal comparison (plant to plant) 
and to use to monitor internal cost fluctuations. A three pronged 
total cost of quality concept, broken down into prevention, appraisal 
and failure costs, was agreed upon. These costs were measured against 
the standard cost of goods produced -- in our case, the standard cost 
of castings. 



PREVENTION COSTS 


Those costs associated with efforts to prevent and correct quality- 
problems, reduce costs and increase customer satisfaction are Prevention. 
Such costs were incurred by product quality activities in these areas: 

I . Quality Planning and Engineering 

A. General administrative reports 

B. Pre-production evaluation and process capability studies 

C. Quality control systems evaluation 

II . Quality Training 

A. Both in plant and outside training programs which include 
quality control personnel and management, seminars, etc. 

III . Corrective Action - Problem Solving 

A. Identification of and correcting the causes of 
defective material through such activities as 
process improvement, quality control audits and 
investigations, and reliability analysis, but not 
including activities concerning product disposition 
on non- conforming material related to scrap, rework, 
and salvage. 

In 1974, .1 per cent of the standard cost of good castings was in- 

cluded in this prevention area. The following year, the area was ex- 
panded when some of the salaried group helped in the production area by 
documenting, evaluating and auditing. The Plant also hired a Quality 
Engineer to deploy their efforts, all increasing the cost percentage to 
. 7 per cent . 


Prevention Costs 


■ Quality planning 

■ Engineering 
■Quality training 

■ Corrective action — 
problem solving 



Expansion of the area continued in 1976 when additional process 
documentation, audits, and evaluations with recommendations were added, 
and a Sand Process Engineer was employed. In 1977, the salaried Quality 
Control group took on still more responsibilities by completing process 
documentation and evaluating test results . 




APPRAISAL COSTS 


All inspection costs, whether they are incurred during the process, 
at final inspection, or in the reliability performance test, are Ap- 
praisal Costs. In 1974 about 2.4 per cent of the quality costs fell into 
this category, and in 1975 that figure rose to 3.0 per cent. Elimi- 
nating some of the in-process inspection in che Mill Room reduced the 
percentage to 2.0 in 1976, a level that held steady in 1977. 


Appraisal Costs 


Inspection 

■ Test purchased material 

■ Receiving inspection 

■ Layout inspection 

■ First piece inspection 

■ Laboratory analysis 
and testing 

■ Product quality audits 

■ Performance /functional 
testing 



FAILURE COSTS 

Failure Costs include the scrap and rework costs both within and 
outside of the Foundry. In 1974 those costs represented about 16.5 per 
cent of the standard cost of good castings, but the quality control 
program was effective in steadily reducing that figure. By the end of 
1975 Failure Costs were down to 14.9 per cent, and were more than cut in 
half in 1976 when they were only 6.5 per cent. Failure Costs for 1977 
were 5.9 per cent. 
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CONCLUSIONS 


The International Harvester cost system for evaluating performance 
at the Indianapolis Foundry has proven to be very effective in showing 
the impact of quality costs as a business expense. Improvements made 
in the total quality cost area are readily apparent on this chart, which 
shows a decrease of 10 per cent in the standard cost of a good casting, 
from 19.1 per cent in 1974 to 9.1 per cent in 1977. This improvement 
has easily paid for the added 16 employes, and in addition saves the 
Company a large amount of money. 


Total Quality Costs 


Sum Total of 
Aforementioned Costs 



Taking a preventative approach to quality problems, rather than 
attempting to inspect quality into the product, seems to be working well 
for International Harvester's Indianapolis Foundry. Cost reduction re- 
sults support management's view that during the four year period from 
1974 - 1977 the Quality Control program really worked. 

That success has pointed out a need for a continued and expanded 
effort in those areas not currently involved with evaluation and cor- 
rection. The Indianapolis Foundry plans to meet that need, and to 
reduce total quality costs further to the 4 to 5 per cent range as soon 
as possible. 
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APPLICATION OF QUALITY TECHNOLOGY TO PUBLIC EDUCATION 


Paul H. Zorger, Ph.D., P.E. 
Consultant 

Annandale, Virginia 22003 


INTRODUCTION 


Background 

Public schools throughout the United States have received negative publicity 
regarding the quality of education provided to the local community. Declining test 
score values and comparative national averages are used as the indicators of the 
decrease in the quality of education provided. Educators generally denounce these 
claims, but investigations and assessments of the causes for such situations generally 
verify that little or no improvements are made which effect the quality of education. 

Over the last thirty year period, the word "quality” has been associated with 
education and school systems as a positive descriptor of public education. Citizens 
are interested in the quality of education available to their children; hence, school 
officials continually use this word as justification for increased yearly funding and 
for requiring bond issues. The resultant costs of public education have now reached 
proportions of available public funds which cannot be met. In the past months, a 
number of Midwest school systems have closed because of the lack of funds. Costs for 
schools within governmental jurisdictions are between 65% and 95% of the total 
governmental budget. Recently it was reported that the cost per pupil for private 
schools was one-half the cost of public education with a higher quality of education 
received . 

How effective are educational systems? On one hand, the citizens observe the 
results of education within their own children and are disillusioned when compared to 
costs. On the other hand, school officials continually offer greater "quality" to 
justify the increased costs. However, this word "quality" has not been defined and is 
used only as a descriptor. 

There have been reports of school systems within this country where citizens no 
longer have a voice in their educational system. They are presented an ultimatum — 
what is to be taught and the cost. Quality is not addressed. Public education is a 
local community responsibility and is under local community control — or is it? 

Definition of "Quality" as Applied to Public Education 

The word "quality" in the English language has implied a qualitative 
characteristic which has assumed many different interpretations. As quality technology 
has advanced, quality has been defined generically and can be quantitatively measured. 
For purposes of this presentation, "quality" is defined as: 

"...that level of education which is acceptable to the local community." 

"Level" is defined by standards developed for each subject and grade level within the 
school system. The word "acceptable" generally is related to economic acceptability. 
This definition satisfys the general model of a quality program — higher standards, 
higher costs; lower standards, lower costs. However there are minimum educational 
standards and there should be costs associated with these minimum educational programs 
which may be acceptable to the citizens within a local community, and which offer each 
student a "basic education" to meet lifetime objectives. The problem with this 
definition however is two fold: (1) there are no detailed educational program 
standards, hence no basis for measurement, and (2) there is no methodology applied to 
measure the uniformity of administration of standards across a school system. This 
combination is the basis for this technical presentation. 

Equal Quality of Education 

Over the past decade turmoil has developed within our school systems over "equal 
quality of education" for every child within a specific governmental jurisdiction. Is 
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equal quality the question, or is "equal educational opportunity" the real question? 
Regardless, the problem is solved by legislation. By racial mixing of students from 
various schools together, legal experts indicate the quality of education will improve 
and that equal educational opportunity will result. Has it? This approach and 
rationale behind these decisions leaves a lot to be desired and there is a real 
question if every student in this country will receive an acceptable quality of 
education or if he will have equal educational opportunity. Today, the quality level 
of education which a child receives is a function of where he resides and which school 
he attends. 

One will admit that school instruction and facilities tend to become segregated 
and polarized within school systems. Admittedly this is wrong; but do we make the 
quality for ALL students better by busing them to other schools and "so called" 
desegregation of the school system? It is implied within this approach that "quality" 
of education and equal educational opportunity will occur through legislation. Has the 
quality of education improved? Indications are that it has not. Has it degraded? 
Indications are that it has. Has a measurement system been developed to measure 
improvement of quality? No. Have educators attempted to develop more detailed 
standards by which to measure the quality of education? Have local educators asked the 
citizens of the local community what level of education is acceptable at a specified 
cost? No! 

Education and educators plod along using the traditional techniques and methods of 
fifty years ago. Our systems are "clogged" with programs which are poorly developed 
and administered, and basic educational courses are suffering because of the "clogged" 
system. Educators should be concerned with improving the quality of basic education 
and be interested in developing a methodology for comparative measurement of the 
quality of various programs. Such an approach would improve the quality of the 
educational system, provide an equal quality of education to all students within a 
community (regardless of the quality level) and improve school-community relations by 
providing the opportunity to all citizens of "having a say" in their educational 
system. 

Educators make the false assumption in analyzing test scores and in explaining the 
results that the quality of education provided in each classroom within the 
jurisdiction is the same, and that test score variations are due to the capabilities of 
the students. There is no effort to assess the cause of the variations nor 
quantitatively measure the quality of the education program offered which would 
validate the underlying assumption that the quality is relatively the same and that the 
range of score values is a true reflection of the range of capabilities of the 
students. If educators desire to interpret test scores and to use them as a basis for 
justifying costs of education, and of gaining community support, they must use valid 
assumptions — that is, that the quality of education administered to all students was 
relatively the same and that each student had equal educational opportunity to learn. 
Only then can test scores be used to indicate that the student has learned in 
accordance with his inherent capability to learn. 

THE APPLICATION OF QUALITY TECHNOLOGY 
Developing Educational Standards 

To establish a basis for quantitative measurement of quality, educational 
standards must be developed. These standards provide the "level" of quality in the 
quality definition. Course and curriculum standards are developed by educational 
"specialists" within the school system. Using a new fourth-grade math program as an 
example, let's examine the general procedure which is commonly used. 

First, the math program is designed by educators schooled in mathematics. The 
program is outlined by the mathematics specialists, specifying the topics, educational 
objectives, and other unique characteristics to be considered. Secondly, media 
specialists provide recommendations for textbooks and other materials to be used. 
Audio-visual specialists provide inputs on A-V uses, materials, and suggested methods 
of presentation. Finally, facility specialists provide information concerning the 
required facilities to be used to enhance the learning process. Instructional 
specialists recommend the most effective methods to be used in presenting the course. 

Thus all of the educational talent required to design and develop the 
instructional rationale, the materials to be used, and the facilities required to 
achieve the educational mathematics program for the fourth-grade has been developed. 
Normally, all fourth-grade teachers are then assembled for an "in-service training 



program" in how to present the program in the most effective manner in each classroom 
within the educational system. 

It can be observed that no mention has been made of the Director of Quality 
Assurance. The Director of Quality does not engage in the design and development of 
the program. He may, if qualified and if asked, provide inputs which could make the 
program easier to administer, but in general he does not contribute to the course 
design. His role is to measure the uniformity or variance from the created standard 
across the entire school system. 

During the in-service training of the teachers, the reliability (repeatability) of 
the developed prototype course may be measured to provide assurance that the 
instruction, materials, and facility requirements are practical, and produce the 
desired objectives within their functional limitations and constraints. The 
reliability of the course is the responsibility of the course designers including the 
responsibility for its achieving the objectives. Course reliability is a 
characteristic of the course design. 

Administering the Course Within the Educational System 

When the course has been designed, developed, tested and teachers trained in 
administering the course, the course is implemented throughout the educational system 
and becomes a part of the "standard" curriculum. As the program is conducted by the 
individual classroom teacher, the role of the Director of Quality Assurance becomes 
apparent. The objective of the Quality Assurance program is to "assure" or provide 
confidence to the Superintendent of Schools, the School Board and the citizens that 
the mathematical program is administered uniformly in accordance with the approved 
standard and that each student has equal opportunity to learn from it in accordance 
with his/her capabilities. The Director of Quality Assurance is responsible for 
assessing or evaluating each classroom and school mathematics program for variance from 
the standard. 



The Director of Quality Assurance collects data on the variance of the application of 
the mathematics program in accordance with the developed standard. 

Analysis of the Collected Data 

Two techniques for measuring variance from a standard or mean (X) value are the 
Range (R) and the Standard Deviation (a) . In a particular school system of 1800 
teachers and 132312 students, test scores in a particular subject by schools had a 
range of from 32 to 71, indicating a large positive and negative variance from the 
standard. If the degree of variance from the program standard (instruction, materials, 
and facilities) were measured and correlated with test scores, there would be a high 
probability that the low socres were the result of a poor instructional and poorly 
administered program. This would be interpreted to mean that students in the classroom 
where the "poor" instruction took place did not have an equal opportunity to learn in 
accordance with their capabilities to learn as students in a class with "good" 
instruction. Wide variance of program administration relates to "poor quality;" small 
variance from the program standards results in "high quality" as described by the 
standard . 
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Test Scores in Mathematics 
At Entrance to Fourth Grade 



Xl x 2 

Test Scores in Mathematics 
At End of Fourth Grade 


If a population of students whose capabilities to learn are normally distributed, and 
they were tested in mathematics at entrance to the fourth grade, and a mean score value 
obtained, the mean score value should increase at completion of the fourth grade if the 
students learned in accordance with their abilities and if the program was administered 
equally across the entire school system (minimum variance — high quality). If there 
is a wide variance in the program administration within a number of classrooms, the 
mean value of the distribution will not shift in a positive direction, this is caused 
by students with a poor teacher not learning in accordance with their capabilities and 
this offsets gains made by students who learn in classrooms with good instruction. 

Educators make the assumption that there is equal quality of program 
administration in each classroom and that mean test score values are not changing 
because of a student’s capability. Of course, this assumption and interpretation is not 
true. To be able to correctly interpret the advancement in learning by administration 
of educational programs requires that a quality program be implemented within the 
school system and that the cause for students not learning be assignable — either to 
capability of the student or to poor educational program administration. Past 
experience and evaluation indicates the latter in a majority of cases. 

ESTABLISHING A QUALITY PROGRAM 


Quality Technology 

Quality Technology is subdivided into three (3) parts: Quality Engineering, 
Quality Control, and Quality Assurance. 

Quality Engineering involves the design and development of a quality system for 
public school systems. The design is performed by professional quality engineers 
knowledgeable in public school operations. Quality engineers would probably not be 
employed by public school systems because of the scarcity of quality engineers and the 
minimal need by public schools. A quality model adaptable to public schools should be 
funded and developed by the National Institutes of Education and made available to all 
public schools. Such a model can be implemented and modified for a specific school 
system by professional quality engineering consultants. The model would include all 
procedures, instructions, evaluation criteria, techniques, etc. required to implement 
and operate a quality program within a public school system. 

Quality Control is the operation of the quality system in accordance with the 
quality system design and the policies established by the school system and the 
community in accordance with the quality definition. Teachers, principals, 
supervisors, directors, associate superintendents, etc. operate the quality program in 
accordance with the design and often a training program involving its operation, 
responsibilities, objectives, etc. School system personnel, by their actions, maintain 
high quality education in accordance with the established standard (level) acceptable 
to the community. 

Quality Assurance provides confidence that the quality system is being operated in 
accordance with the design. Quality Assurance is the responsibility of the Director of 
Quality Assurance for the school system. He continually conducts audits and 
assessments of the quality of education and provides assurance (confidence) to the 
Superintendent of Schools that a uniform quality level of educational programs is 
administered across the educational system. The Director of Quality Assurance 
continaully supervises the audit, assessment, evaluation and measurement techniques 
within the school system and recommends corrective action to the Superintendent, when 
indicated by quality audits. 



The Role of the Director of Quality Assurance 


The role of the Director of Quality Assurance in a public school system is one of 
organizing, implementing and operating a quality program within the school system. The 
quality program should extend across the entire jurisdiction and may include 
application to the complete range of school activities, i.e. education, administration, 
construction, etc. depending upon the objectives and authority given to the Director of 
Quality. 

Initially the Director of Quality should establish an operating rationale. This 
rationale is then converted into operating procedures which describe "how" the quality 
program functions and relates the techniques and methods to be used in conducting the 
quality assurance activities. Procedures, techniques, methods, organization, 
rationale, flow charts, data, etc. are all documented in a Quality Manual for the 
school system. It should be sufficiently detailed that it provides for continuous 
operation of the Quality Program regardless of personnel turnovers and/or changes in 
the school system organization. 

The primary role of the Director of Quality is to measure quantitatively the 
variance of educational instruction, materials and facilities throughout the school 
system, between schools, and between all classrooms. Based upon the variance data 
obtained, the Director of Quality documents the conditions, analyzes the data, and 
makes recommendations to the Superintendent of Schools for any required corrective 
action. For example, data collected may indicate data points at extremes from the mean 
(X) value of the distribution and an analysis indicates that poor instruction is the 
most probable cause. The Director of Quality should indicate that a possible problem 
exists. It is the responsibility of the Superintendent of Schools to take the 
corrective action, probably providing an in-service training program to improve the 
instruction. The major point is that the "Director of Quality" collects quality data, 
measures the variances across the school system, and recommends corrective action where 
and when indicated, but he does not become involved in the operation of the school 
system. Operating in an unbiased manner and using scientific approaches, the Director 
of Quality Assurance implies a degree of confidence to the school administration and to 
the citizens in their educational system and programs. 

If the quality program extends into other school related programs such as 
administration, facilities, construction, etc. the Director of Quality should establish 
a quality program to control variances of activities within these areas in accordance 
with acceptable product standards. The primary activity of the Director of Quality 
Assurance is to provide confidence to the School Superintendent, School Board and the 
citizen taxpayer that the quality level to which they are entitled is being achieved 
and maintained. 


PROBLEMS ASSOCIATED WITH IMPLEMENTING QUALITY PROGRAMS 
Design and Implementation of Quality Programs 

The development of quality program models for a specific school system is very 
expensive for an individual school system. If this concept is worthwhile and 
applicable to ALL public school systems, a model should be available which can be 
modified for specific school system application. Such a function is the responsibility 
of the National Institutes of Education and the Department of Health, Education and 
Welfare. If such a model were developed, the cost to implement a quality program for a 
school system would be substantially reduced. Such quality models would be developed 
by professional quality engineers. 

Understanding the Quality Concept 

If the quality concept is to be applied to public school systems, the concept 
together with the advantages must be made known to public school educators and 
administrators. Efforts to acquaint school administrators can be enhanced by the 
concept and advantages being described in publications which are directed to the school 
administrator. Organizations such as the American Association of School Administrators 
are excellent for disseminating the information. The American Society for Quality 
Control (ASQC) is the professional organization associated with the Quality profession 
and should begin to broaden the application of Quality Technology to other fields such 
as Education and Training. 



Development of Quality Assurance Directors 


If a Superintendent of Schools recognizes the advantage of a quality system, and 
if a model is available for implementation, the third major program ingredient is 
required. This ,1s a Quality Assurance professional to operate the quality program 
within the school system. The approach being pursued to date proposes to develop 
Directors of Quality Assurance within a Master's Degree program within a School of 
Education within a University. The development of such professionals would be similar 
to developing public school administrators with MS degrees in Education. Some graduate 
schools of Education have expressed interest in developing such professional 
development programs. 


SUMMARY 


The Concept 

The concept of applying quality technology to public education is a valid and 
worthwhile approach to solving some modern sociological and economic problems. In a 
modern society where everyone should have an equal opportunity to learn in accordance 
with their capabilities, it is mandatory that a system be established to achieve this 
democratic objective. Current public education programs are not valid in their 
assumptions. Trends seem to indicate that unmeasurable education systems can only be 
improved with increased costs, and these unrealistic approaches are reaching maximum 
limits of the American taxpayer. Therefore, the application of quality technology to 
public education can both improve the local communities economically as well as improve 
the educational system itself. 

Implementation of the Concep t 

To implement quality programs within educational systems requires: 

• Dissemination of the quality concept to public school superintendents and 
administrators . 

• Development of a quality system model for Public School Systems funded by the 
National Institutes of Education. 

• The development of Directors of Quality Assurance for public school systems 
through a graduate program in Schools of Education of our Colleges and 
Universities . 

Conclusion 


Much interest has been expressed in this concept. Some school systems have 
already implemented Quality programs — some wrongly, but it is a start. Parents and 
teachers, school boards, and state public education officers have expressed interest in 
the approach to improving educational programs and opportunities. The American Society 
for Quality Control can assist in such a program by dissemination of information and 
providing assistance to American public school systems. Who can better promote the 
improvement of Quality of Education than the professional organization whose purpose is 
the Management, Engineering, and Scientific Aspects of Quality and Reliability — 
applied to Public Education. 


LCS 


320 : 70:000 



INITIATING PRODUCTIVITY MEASUREMENT SYSTEMS IN MIS 

D. Scott Sink, Field Director 
The Ohio State University Productivity Research Group 
Columbus, Ohio 43210 


ABSTRACT 

Participative methods involving structured group processes have been 
utilized to begin the process of designing productivity measurement 
systems for computer based management information systems. The research 
indicates that the Nominal Group and Delphi methods, used for this pur- 
pose serve to establish favorable bases for completing the creation of 
productivity measurement systems. The research further indicates the. 
composition of measures elicited by the participants suggest interesting 
contrasts between "traditional” efficiency measures and "non-traditional” 
effectiveness measures. Current research efforts stress the importance 
of strategies designed to operationalize the measures generated by 
participative methods. 


FUNDAMENTAL ISSUES 

Productivity, in general, is an elusive concept. It suffers from 
a historical background founded in misunderstanding, confusion, myopia, 
and rather uncreative and ineffective approaches to definition, measure- 
ment, and improvement. Traditional definitions, measurement techniques, 
and improvement strategies have neglected several important aspects of 
productivity; 


• Productivity issues, for almost any system 
under study, are complex and thus multiple 
measures are required to convey the total 
picture 

• Productivity means different things to 
different people and thus the more view- 
points involved, the more representative 
the productivity measurement system will be 

• Productivity measurement and improvement can 
be attained through efforts to find the 
"right mix" of technical and behavioral, 
techniques and methods for application in 
areas of concern. 

Administrative computing and information (ACI) services have. evolved 
with considerable rapidity, have undergone continuing evolution driven 
by high innovation rates in the underlying technology, and have quite 
often been imperfectly assimilated into the operating and management 
structure of the organizations they serve. With this in mind, research 
concerned with productivity measurement systems has focused on develop- 
ment of methods which incorporate the three above mentioned asspects. 


*This research was supported by the National 
Science Foundation RANN Program, Division of 
Advanced Productivity Research, Grant No. 

APR 75-20561. 
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INITIATING DESIGN FOR PRODUCTIVITY MEASUREMENT SYSTEMS 

The design and management activities for productivity measurement 
systems in ACI services should include the following: 

1. Generating and weighting measures of productivity 
for the ACI system, utilizing all of the relevant 
viewpoints within the organization. These measures 
may he thought of as comprising a vector of meas- 
ures which attempt to assess the efficiency and 
effectiveness of ACI systems from the viewpoints 

of 

a. ACI service staff and operating 
management ; 

h. the various clients of the service, 
including those who receive it's 
information and documentary output, 
as well as managers whose decision 
making it seeks to enhance; and 

c. the overall organizational objectives 
as embodied in the planning and 
decision making of top managers 
who determine ACI budgets and policies; 

2. Specifying indicants of performance which could be 
used to assess the states of the selected vector 
of productivity measures (ie. an indicant is that 
which is used to assess the performance relative 
to a particular measure, for example, turnaround 
time as an indicant for user satisfaction) ; 

3. Structuring the measures and indicants into a 
measurement system which determines which are 
presently available and which will require further 
technical assistance to bring into being; 

4. Creating the necessary measurement instruments 
and methods required to bring the indicants into 
being leading to operationalization of the measures; 

5. Executing a carefully designed program of imple- 
mentation which will hopefully lead to an opera- 
ting productivity measurement system available 
as an organization and management resource. 

The "Right Mix" of Technical and Behavioral Techniques 

Recent research and literature in the organizational behavior field 
has focused on the contingent nature of structure and process within 
organizations . (2) Concerns are raised relative to the influences of 
external factors(ie. environment), and internal factors(ie. structural 
features: size, shape, etc. and process features: authority, control, 

leadership, nature of group processes, etc.) on the performance of the 
organization , group, and individual. The basic concerns and interests 
lie with the nature of interrelationships among the factors. 

Consideration of this research and literature might lead an organ- 
ization, manager, or researcher to consider process issues relative to 
the creation of productivity measurement systems. The nature of prod- 
uctivity measurement systems as mentioned previously suggests that the 
process used to develop the system could be as important as the system 
itself, if not more so. 

The classical approach to productivity measurement uses what might 
be called the consultation mode. This produces an expert-imposed system 
for measuring productivity. Results have been obtained with this ap- 



proach, however, long term effective utilization of these systems appear 
to be of limited success. These traditional expert-imposed methods for 
measuring and improving productivity have focused primarily on technical 
solutions, ie., making operations more efficient. Efficiency measures 
tend to be relatively straight forward, inexpensive, 'objective', un- 
obtrusive, and suited to portray only a very narrow view of the services 
'real productivity'. Even for these measures, knowledge and communica- 
tion of the productivity measurement has been limited to a few number of 
data center personnel. On the other hand, effectiveness measures tend 
to be relatively expensive, poorly understood, controversial, qualita- 
tive rather than quantitative, and 'subjective' rather than 'objective'. 
They are a strong function of one's perception of the purpose of the 
organization and they represent attempts to portray the 'real producti- 
vity' of the system. 

An alternative to the consultation mode is a participative approach. 
A substantial body of literature and experience relative to partipation 
and group processes suggests that there exists a great amount of un- 
tapped resources within organizations in the form of its employees. 
Groups can be used to make decisions, solve problems, generate alterna- 
tives, plan, design tasks, set goals and objectives, etc. Groups pro- 
vide at least the following functional consequences : 

• groups bring together knowledge and skills not 
possessed by a single individual 

• the product of a group is more likely to be 
accepted by those who must act on it, than 
is the product of an individual 

•there is a "group effect" through which group 
members learn from each other, stimulate each 
other, and supplement each other's knowledge 
and skills. The product of the group is in 
this sense greater than the sum of the indi- 
vidual contributions. 

Groups, however, can also be accompanied by dysfunctional consequences 
such as time and resource consumption, inefficient operation, failure 
to act , etc . 

The activities necessary to generate a productivity measurement 
system can be assessed relative to their inherent potential for use of 
group versus individual oriented strategies. This particular research 
believed there was a strong rationale for utilizing structured group 
processes for the activities of generating and weighting measures and 
indicants . 

Two of the most widely known examples of structured group processes 
are the Nominal Group Technique and the Delphi Method. (3) The struc- 
tured group process approach gives the consultant a low profile as a 
provider of methodology and strategy, rather than as a substantive re- 
source for the group. Both the Nominal Group Technique and the Delphi 
Method consist of a set of clear, well defined activities. The process 
consultant may manipulate design variables such as composition of the 
group, size of the group, and nature of task statement in order to en- 
hance the effectiveness of the process relative to the particular situa- 
tion. 


This research has investigated the application of two structured 
group processes, the Nominal Group and the Delphi methods. The work has 
been carried out at four large public sector organizations ; two state 
governments, a city government and a public corporation employing some 
30,000 people. Approximately 300 persons have been involved, investing 
roughly 1200 person-hours in the initiation of a productivity measure- 
ment system for ACI services. 

The final steps in the process were not included in the original 
study(ie. steps 3»^ and 5)> however, current research is concentrating 
on technical methods which can be utilized to accomplish these activi- 



ties, which extend beyond the initiation of productivity measurement 
systems. The current research will be discussed, briefly, later in this 
paper . 

The Nominal Group Technique 

The Nominal Group Technique (NGT) is a four-phase process. The 
participants are physically present in groups of eight to twelve and the 
session is controlled by a process consultant or facilitator. 

Following an opening introduction in which the purposes of the ses- 
sion are outlined, participants are presented a carefully worded task 
statement. For example, Task I in all of our studies was, "Please list 
below the important measures ( indicators , criteria) of the productivity, 
effectiveness, or contribution of computing and information services 
from the point of view of your department or division." The group mem- 
bers are then instructed to write on the sheet provided, their responses 
to the task statement. The first phase is called silent generation and 
typically takes about ten minutes. 

Next comes the round robin phase. The facilitator calls on parti- 
cipants one-by-one to state one of the responses he or she has written. 
Participants may pass at any time and join in on any subsequent round. 

A participant may propose only one measure at a time and either the 
facilitator or an assistant records each item as it is offered. (A large 
easel is ideal since all items must be in full view of the group for the 
remainder of the session.) The only discussion allowed is between the 
facilitator and the participant who proposes the item and it is limited 
to seeking a concise rephrasing for ease of recording. Participants are 
encouraged to add items to their personal list should new ones occur to 
them during the round robin. 

The third phase is called clarification . Once all items have been 
recorded, the facilitator goes over each one at a time to ascertain that 
all participants understand the measures which have been recorded. Any 
participant may offer clarification or may suggest combination of simi- 
lar items, however, no evaluation is permitted. 

In phase four, voting and ranking , participants are provided with 
eight blank cards. Each must now select eight preferred measures and 
write them, one per card. Participants then spread the eight cards such 
that they can be viewed simultaneously. Working alone, each selects the 
single most preferred item and writes the score 8 on the card, and. puts 
it aside. Of the remaining seven cards, the least preferred measure is 
selected and scored, 1. This iterative process continues until all are 
scored . 

During a short break results are compiled and the scores are record- 
ed beside the items on the clarified list at the front of the room. The 
resulting consensus measures are discussed. The entire process should 
require about two hours. 

The above process is used to generate and weight measures and indi- 
cants. However, specification of indicants does not require phase four 
of the NGT process. 

The Delphi Method 

The Delphi is a methodology for the systematic solicitation and 
collation of judgements on a particular topic through a set of carefully 
designed sequential questionnaires interspersed with summarized informa- 
tion and feedback of opinions derived from earlier responses. 

These sequential questionnaires are used to obtain information or 
opinions from a group of experts; the respondents. One important fea- 
ture of the Delphi is that the participants never meet face-to-face, 
since the questionnaires are usually mailed. For this reason, group 
members retain their anonymity throughout the process and any negative 
attributes of most group techniques, ie . dominance, interpersonal con- 
flict, are avoided. Additionally, unlike the Nominal Group Technique 



the Delphi does not require movement of people in order to accomplish 
the task objectives. The Delphi represents a tradeoff of cost and time 
relative to the Nominal Group Technique. The Delphi can usually accomo- 
date larger numbers of participants at a lower cost, whereas the Nominal 
Group Technique can usually be completed in less time. 

Both the Delphi and the Nominal Group Technique were used to 
accomplish the generation and weighting steps. That is, the task state- 
ments for both processes were identical, only the nature of the struc- 
tured group process diff erred. 

The typical results from the research sites can be seen in figure 

1 . 


Results of Activity 1 



Results of Activity 2 

PRODUCTIVITY MEASURES: 

Votes 

Received 

Total 

"HOW TO MEASURE, HOW TO 
ASSESS, WHAT ARE THE 
INDICANTS?" 

The degree to which the 
data center meets the 
defined needs of the users 

Responsiveness of the data 
center to problems, com- 
plaints and inquiries 

?-7-7-8 

6-6-8-8 

6-6-7 -5 
6-7-6-5 

57 

48 

Number and type of comp- 
laints; Percentage of in- 
formation that is rele- 
vant ; etc . 

Percentage of repeat 
complaints; Time from com- 
plaint to solution; etc. 

Degree to which data 
center coordinates and 
communicates with vendors, 
users, and within the 
data center itself 

6-5-5-5 

8-8-4-5 

46 

Percentage of users aware 
of data center capabili- 
ties; Number of user-data 
center meetings; etc. 

Ability to update and 
keep current the infor- 
mation system 

5-5- 7-4 

4_3_4_4 

36 

Time required to update; 
data entry capacity; etc. 

Timeliness of output 
and information 

etc . 

4-4-3-3 

2-2 

18 

Time needed versus time 
received; percentage of 
user deadlines met; etc. 
etc . 


Figure 1 . --Exemplary Consensus Measures 
from Research Sites 


WHAT CAN BE EXPECTED FROM APPLICATION OF 
STRUCTURED GROUP PROCESSES FOR INITIATION OF PRODUCTIVITY 
MEASUREMENT SYSTEMS ? 

Research results indicate that generation and weighting of meas- 
ures and indicants can be successfully accomplished utilizing structured 
group processes. The overall quantity and quality of the measures is 
comparable to any expert generated list. However, the expected advan- 
tages of participatory approaches cannot fully be acquired unless the 
organization attempts to complete the activities as outlined earlier. 

It is believed that certain factors contribute to an organization's in- 
clination to continue the development of productivity measurement sys- 
tems beyond the initiation stages. Probably the two most significant 
factors are the general salience of productivity assessment and the 
stage of growth of the ACI system. (1,4) The previous research also pro- 
duced several important insights relative to the nature of productivity 



measures generated from participative methods: 

-there appeared to be considerable ambiguity 
within organizations relative to the nature 
of or mere existence of productivity measure- 
ment systems. There was no clear productivity 
measurement program in any of the organizations 
studied, although certain "traditional” effic- 
iency measures were monitored ( ie . downtime, 
turnaround time, reliability, etc.). 

-the consensus measures across research 
sites reflected a heavy emphasis on what 
might be termed "non-traditional" effec- 
tiveness measures .( ie . user satisfaction, 
degree to which data center meets users 
needs, etc.) Approximately 50$ of the 
measures were of this category, and those 
measures were most frequently among the top 
four consensus measures for each partici- 
pating group. 

-approximately one-half of the indicants 
(activity 2) were judged to be ready for 
incorporation into a measurement system 
with little or no modification (that is 
they were operational) . 

-much of the concern for conflict and lack 
of consensus regarding measures and indicants 
for productivity appears to be attributable 
to point-of-view discrepancies arising from 
differing positions within the organization. 

The group process appears to have a remark- 
able ability to provide an opportunity for 
heterogeneous groups to reach consensus re- 
garding the components of a productivity 
measurement system. The process can addi- 
tionally be used to separate and prioritize 
measures relative to various perspectives 
within the organization . 

-basic difficulties regarding taking a 
productivity measurement system beyond 
generation and weighting appears to be 
attributable to hierarchal structure of 
measure-indicant relationships. For 
instance, user satisfaction can be a meas- 
ure for which turnaround time may be considered 
an indicant. However, turnaround time may 
be considered an independent measure in itself. 

It appears that these major issues need to 
be researched. 

BEYOND INITIATING PRODUCTIVITY MEASUREMENT SYSTEMS 

Generation of consensus measures and indicants which represent 
timely organizational values, concerns, conflicts, agreements , etc . is a 
major step in the direction of generation of a useful and effective pro- 
ductivity measurement system. It is a step which few organizations have 
taken or are capable of taking. Therefore, we believe this research has 
been a contribution to the field of productivity research as well as to 
applied productivity concepts. 

An organization's willingness and interest in assessing and improv- 
ing productivity is a necessary ingrediant for generation of a completed 
productivity measurement system. Assuming this ingrediant exists, it is 
believed that utilization of the methodology described in correlate re- 
search publications (l) will provide any organization with the basis for 
a productivity measurement system for ACI services. Once completed with 



initial activities any organization faces implementation issues. The 
organization may choose to; do nothing else, utilize the results in a 
segmented fashion (ie. benefits and applications accrue to individual 
departments in form of ideas for improving communication between users 
and providers, ideas for centralization versus decentralization deci- 
sions, etc.), utilize the results to form the foundation for a produc- 
tivity assessment system, utilize the results to form and derive a pro- 
ductivity improvement system. The activities required to utilize the 
results of initial activities for development of more sophisticated pro- 
ductivity assessment or improvement systems have not been clearly formu- 
lated as of yet. However, current research has begun to focus on use of 
multi-attribute utility models in order to aggregate productivity 
measure assessment into one number which might be thought of as a pro- 
ductivity index. Other research concepts focus on the need to consider 
the productivity measures and assessment indicants as a non-collapsable 
vector of productivity. These concepts infer that decision makers 
would achieve insight relative to areas for improvement as a result of 
observing the evaluation of the measures in a prioritized vector format. 

It is likely that the most successful strategy for productivity 
assessment and improvement will be one that is tailored to each organi- 
zation's special needs. Therefore, much of the present concern is di- 
rected at identifying methods which allow individual organizations to 
design their own specific productivity assessment and improvement system, 
The foundation for progress in the area of productivity assessment and 
improvement has been laid. It is desirable for industry, government , 
and academia to continue to strive to work together in this endeavor. 
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THE MANAGEMENT INSTINCT 


Harry E, Williams 

Director, Quality Assurance & Reliability 
Electronic Memories & Magnetics Corp. 
Hawthorne, CA 90250 


Management is more an art than a science, but the effective 
manager is both a practicing social scientist and an artist. He 
is sharply differentiated from the scientist or artist because 
the scientist observes and records his findings, while the manager 
goes one step farther and takes action. The artist need only 
express his thoughts or feelings and he can ignore public taste, 
but the manager, working with other people's capital, other people’s 
labor, and other people's material must consider the public's 
opinions in order to make a profit and survive. 

Intelligence, good judgement, analytical ability, energy, etc., 
are all important to managerial success. They are also important 
to success in all professions. 

A recent report contained a consensus on managerial traits. 

They were: 1) initiative, assumption of responsibility and leader- 
ship, 2) job knowledge and skill, 3) dependability, thoroughness 
and follow-through, 4) getting along with people, 5) stability 
under pressure, and 6) fine personal qualities and work habits. 

Another magazine listed the five most needed traits as: 

1) ability to be flexible and adapt to accelerated change, 

2) ability to be imaginative and to innovate, 3) proficiency in 
controlling and reducing expenses, 4) ability to mobilize and 
motivate men, 5) skill in coordinating and correlating forces 
within and outside your company. 

The legal mind is committed to the abstract ideal of justice. 

It is interested primarily in what and how something happened, and 
secondarily, in why it happened. 

The scientist is dedicated to a systematic knowledge of the 
general laws governing the physical or material world. 

The religious individual is absorbed with spiritual values 
and the relation of man to a supernatural being or to an eternal 
principle . 

The educator is interested in developing the special and 
general abilities of the mind by imparting particular knowledge 
or skills . 

The businessman-owner is concerned primarily with profit. 

The engineer working within the organization is involved 
primarily with creating a specific product. He is not normally 
concerned with other people's problems or with the larger problems 
of the organization. 

The stockholder is primarily interested in the organization 
as an investment. If the growth and yield of his stock is satis- 
factory, he is content. The internal workings of the organization 
only concern him when they affect his investment. 
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To the manager, however, the organization is an end in itself. 
He is totally committed to it. He gives his loyalty in return for 
the power, prestige, security, and money it gives to him. 

The organization is important because it exists, he may dis- 
agree with specific policies or with people within the organization 
but he will try to effect changes within the existing structure and 
will not attempt to destroy it. There is in this attitude towards 
the organization, a potential source of trouble; for the manager is 
sometimes torn between his own personal values and his loyalty to 
the organization when its values conflict with his. 

One cannot manage effectively without power. Yet the exercise 
of power tends to alienate people. The effective manager will use 
his power discreetly and judiciously, never employing more than the 
situation demands, yet never hesitating to use it fully when appro- 
priate. He accomplishes his will by persuasion, suggestion, exampl 
but he knows, and his subordinates know, that such techniques are 
effective, at least partially, because of the power of his position 

Since the manager's commitment is to the organization, he must 
use people as means to an end, that is, to effect organizational 
goals. He may temper this harsh reality with tact and discretion 
but he knows that the organization comes first and that the individ 
ual is expendable. Such a realization is a breeding ground for 
resentment and poor morale, and he may suffer strong feelings of 
guilt if he is capable of genuine emotional sympathy and if forced 
to sacrifice an individual for the company's good. This indicates 
that probably the most successful top executives are character- 
ized by a basic emotional coldness with an external ability to get 
along with almost anyone. 

The manager runs the risk of perpetuating his own errors 
because of his authority. He needs to have sufficient power to 
enforce his decisions; but, he must always keep in mind that many 
things are done his way because of this authority and not always 
because he is right. He should not only be able to evaluate criti- 
cism leveled at him by others, but should anticipate such criticism 
by objectively examining his own actions on a continuous basis. 

Self-confidence, aggressiveness, and a strong personality are 
definitely managerial assets, but he must be careful of stifling 
his subordinates' initiative. He should realize that by virtue of 
his position, he is the authoritative father-figure to his subordi- 
nates, and, as the father, wishes his sons to grow up strong and 
independent. The manager, wishing to develop his subordinates to 
their fullest potential, must let them develop confidence. 

This emotional atmosphere enables him to detect potential 
problems. Having utilized his sensitivity to determine where the 
problems lie, he must use his judgement and take action on the 
basis of what is best for the organization. In other words, he may 
have to act ruthlessly. 

The conflict between personal ambition, loyalties, and ethics 
and organizational demands may lead to strong feelings of guilt, 
anxiety, tension, and frustration. For a man whose loyalty is to 
the organization may have to act against the dictates of his temp- 
erament, his sympathies, or his values when the occasion arises. 

A manager may be deeply involved, both emotionally and career- 
wise, with a company that has provided him with meaningful work, 
material rewards, good friends, and psychological sustenance. Then 
another company offers him a job with far greater rewards and poten 
tial. His loyalty urges one course of action while obligation to 
his family and his own future urges another. 



Use of people tends to dull the sensitivity to people's 
"private" feelings. Such human worries as love, death, aging, 
insecurity, morals, etc., are not of interest to him unless they 
affect organization in some way. His use of people in the name of 
organizational goals, tends to restrict his concern for the problems 
of others when they intrude upon organizational efficiency. 

He tends to be uncomfortable with broad social problems which 
do not lend themselves to quick, direct solutions. He tends to 
favor the practical, down-to-earth, immediate solutions over the 
theoretical ones. Actually, the goal and resul ts-or iented manager 
prefers the imperfect, short-range, interim answer to waiting for 
the ideal, long-range one. 

Such an attitude is forced upon him by his job. His work is 
never done. The problems that confront him are recurrent; he solves 
one problem so that he may go on to the next. He is engaged in an 
endless struggle with the powerfull disruptive forces which are 
always active in an organization -- personal animosities, jealousies, 
rivalries, misunderstandings, and conflicting interests -- and must 
often content himself with patchwork solutions to the problems they 
generate. He must meet his quota, produce a profit, or face the 
company's equivalent of execution - that is, termination. 

The pragmatic approach can be considered a managerial strength 
insofar as it provides a way of coping with a multiplicity of prob- 
lems; but it has severe limitations and can create its own problems. 
In the haste and willingness to compromise in finding solutions, he 
often fails to deal with the source of the problem and, by this tech- 
nique, insures the problem's recurrence. 

The importance of the managerial function is incontestable. The 
increasing complexity of the structures in our society, business and 
otherwise, has created a need for men who can handle a complicated 
mix of men, money, and time in achieving organizational goals. Aside 
from the basic prerequisite of making a profit for the organization, 
what contributions does the manager make to the organization and to 
society? Ultimately, it is his effort which determines whether a 
company will prosper. In a society such as ours, which depends 
primarily upon the wealth generated by the business sector, the 
status of the company influences, for better or worse, the status of 
the entire society. 
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QA FOR SMALL PROGRAMS IN A LARGE INDUSTRY 


Henry N . Lange 

Director - Quality Assurance 
McDonnell Douglas Astronautics Company 
Huntington Beach, California 926^+7 


STRICT QUALITY CONTROL SUPPORTS BIG MILITARY AND SPACE PROJECTS 

For a long time we have been in the habit of reacting to government contract 
requirements for quality control. The government quality specifications have gone 
through an evolution of ever-widening scope and increasing detail. In this context 
we have built up procedures that meet the requirements of MIL-I-U5208A , MIL-Q- 9858 A, 

NHB 5300. H (lB), and NHB 5300. U (lC). Additionally, the government agencies have 
refined the requirements for individual projects or product areas, as with NASA’s 
NHB 5300. U (lD-l) for Space Shuttle and SAMSO STD 73-5B for Air Force space and missile 
programs . 

We at McDonnell Douglas are deeply engaged in the development of highly complex 
product systems that demand intensive quality control. We regard the Military Services 
and NASA as primary market areas and must maintain the ability to meet their require- 
ments. At the same time, we are busy with a wide variety of smaller projects. Some 
of these are commercially oriented. Some are for new agencies of the federal govern- 
ment, such as the Department of Energy (DOE). 

R&D AND LIMITED PRODUCTION CALL FOR A CHANGE IN PACE 

The big military and NASA projects that make the news do not appear overnight. 
Contractors who qualify to handle them have generally been through a long sequence of 
research and development activity that does not warrant the full measure of quality 
control. Recognizing this, the Department of Defense in recent years has put great 
official emphasis on tailoring quality requirements to the needs of the particular pro- 
ject at its particular stage of development. But it will take considerable time for 
these policies, such as in DOD Directives 1+120.21 and 1+155.1, to filter down into spe- 
cific contracts. And, in the meantime, we are continuously engaged in a number of 
R and D programs in different stages of development. 

What kind of quality program should the sophisticated Quality Assurance organiza- 
tion have for these smaller programs that we must handle concurrently with the large- 
scale projects. We cannot allow ourselves the luxury of the elaborate disciplines 
devised to get men on the moon. That is why we have had to devise methods with more 
latitude than is permitted by the procedures and formal documentation associated with 
large, complex projects. 

Generally the small jobs are just as competitive as the big ones. That is to say, 
we must bid for them in competition with other firms, so we'd like to determine what 
control is essential and bid to provide that and no more. But we cannot economically 
indulge in a different QA system on each one of the scores of small R&D or commercial 
programs that are active at any one time. Such an approach multiplies management cost 
and causes confusion, lost motion, delay and waste in the work areas. 

COMMON OBJECTIVES PROVIDE GUIDELINES FOR THE SMALL PROGRAM 

At McDonnell Douglas Astronautics we have been learning to tailor our quality 
program to the level of control necessary for the smaller projects. Several examples 
of this are presented in this paper. The approach for different kinds of projects is 
based on some common objectives which can be summarized to provide guidelines as 
follows : 

1. In a test or R&D effort, where the budget is always tight, sufficient discipline 

must be exercised that we run all the tests that are essential and that we test 

the right things the right way. We must make sure that we get what we ordered. 

We want to know what it is we've tested. The definitions must be good enough that 

someone else can repeat the tests and get the same results. Otherwise the tests 
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are not meaningful. Not every element of the design need be monitored, but char- 
acteristics that are significant in determining performance have to be pinned down. 
A first concern, then, is with how we'll define the product or work, how we'll 
order the work done, how we'll keep track of changes, and how we'll record the 
results . 

2. In a limited production effort, for relatively simple products, we must exert suf- 
ficient discipline that we can be sure of meeting the customer's expectations. 

We must know that the design is adequate to meet our commitments. We want to know 
that the product is sufficiently like the design to give the customer what he rea- 
sonably expects — no more and no less. The nature of the product, its intended 
purpose and the customer investment it represents will be major determinants in 
sizing the control program. 

3. Finally, in any effort, we must exert enough discipline to optimize the cost to 
ourselves and to meet important schedules. The area where most latitude is avail- 
able is in choosing the time, place, frequency and depth of inspection. Should we 
check at source, on receipt, during fabrication or processing, or just at the end 
of the line? Can we control effectively by sampling? Can we omit inspection 
entirely? The risks we can afford to take and those we can't will determine the 
answers to these questions. For example, elimination of in-process inspections 
may be a reasonable alternative if it can be done without significantly adding 
risk. Another alternative might be the delegation of certain verifications to 
Manufacturing . 

Certain cares must be taken in tailoring a control system. It is important to 
recognize that the small programs benefit from a number of standard disciplines that 
are maintained for major space, missile and aircraft programs. Many of these can be 
applied selectively, but a few must remain standard. When we are looking for short 
cuts and setting up streamlined procedures, we must be careful that we don't inadver- 
tently by-pass an essential control that might be taken for granted. First and most 
obvious of these is the calibration system. Any measurement made for the purpose of 
controlling, evaluating, or verifying what we do or make is wasted if we cannot trust 
it to some degree of accuracy. Another area to be safeguarded is the training and 
certification program for specialized skills. It's not only likely to yield specious 
results, but it may be downright dangerous to allow untrained personnel to perform 
certain inspections, tests or verifications. 

Now let us look at a few examples of how the full blown quality program has been 
scaled down to the needs of individual smaller projects . 

SMALL COUNTERPART TO A MAJOR PRODUCTION PROGRAM 

CYROANCHOR is the McDonnell Douglas trademark for an ingenius heat pipe used to 
stabilize the soil in arctic and subarctic regions. In such areas temperatures can 
range from -T0°F or below in the dark of winter to as high as +100°F in the unending 
day of full summer. During the thaw, we find an active layer above the permafrost 
(meaning the ever-frozen subsoil). Power poles and pilings may be thrust upward and 
about in the unstable conditions of repeated thaw and refreezing during a change of 
seasons. The resulting effect on supported structures can be catastrophic. CRYO- 
ANCHORS are designed to remove heat from the frozen soil during the winter, further 
lowering its temperature and enlarging the frozen part of the unstable zone so that 
it will endure through the brief but intense summer heat . Over 120 ,000 CRYOANCHORS 
have been installed to stabilize pilings that support the Alaska pipeline as shown 
in Figure 1. These are heavy steel pipes containing liquid ammonia, each one topped 
by a press-fit aluminum radiator to dissipate heat . 

Critical characteristics for top performance of CRYOANCHORS are the interior sur- 
face of the pipe, its texture and cleanliness, quantity and purity of the ammonia, and 
the soundness of welds and materials to secure the pipe against leakage for many years. 
The quality control program for a high-production operation at our Tulsa plant was 
carefully designed, with sampling plans for inspection and test precisely adjusted to 
assure a contracted reliability goal with known confidence. Verifications were made 
and recorded by full-time inspectors at each pre-planned inspection point. 

In addition to the pipeline job, we have pursued a quite different market in sup- 
plying orders for lighter, more versatile, all-aluminum CRYOANCHORS to stabilize the 
soil under individual buildings and other structures. Runs are short, running from 
as few as six to perhaps 100 pieces for a single contract. The design is highly 


flexible, being tailored to the particular 
application. For this reason, the job is 
done in the experimental shops at Hunting- 
ton Beach, close to the sales and engine- 
ering headquarters. Typical of the output 
today is the installation near Fairbanks 
under a satellite-tracking Telemetry Sta- 
tion built for the European Space Research 
Organization. See Figure 2. 

For the intermittent short runs of 
aluminum CRYOANCHORS , we took a different 
approach to quality control. Inspection 
and proofing of tools and processes were 
as strict as for the big production job at 
Tulsa. Two first articles were made and 
completely tested under the surveillance 
of a quality control manager. Thereafter 
the inspection function was wholly dele- 
gated to a senior mechanic in the experi- 
mental shops who is fully responsible 
for all operations , performing most of 
them himself with intermittent support in 
tasks that require more than one set of 
hands. Critical verifications are docu- 
mented in the simplified work order that 
serves as a check list, and the senior 
mechanic makes all the entries as tasks 
are performed. A QA professional makes 
spot visits for surveillance and performs 
final review of the records when examin- 
ing the completed lot for shipment. One 
specimen is then taken at random from 
each lot and functionally tested to 
verify performance. 



Figure 1. Cryoanchors for Alaska Pipeline 




The operation is small enough today that we do not wish to continue devoting the 
space and talent in the experimental shops to this diminutive but technically sound 
and proven bit of business. Because of its potential, however, intermittent produc- 
tion is performed while we actively seek to license manufacture of CRYOANCHORS by a 
capable firm that will develop the market. 

DEVELOPMENT AND DELIVERY OF ONE OF A KIND 

Optical sensors find many applications in space and weapon programs. In our 
space simulation laboratory, over the course of several R&D projects, we developed a 
unique capability to test such sensors in highly specialized environments. Concur- 
rently, in laboratories and facilities at widespread points in the country, the ability 
was developed to generate other environments important for individual sensors . But 
our sensor testor was fixed at Huntington Beach. Because of the nature of the envir- 
onments to be induced and because of the size of some environmental test facilities, 
one customer concluded that we should make a sensor tester that could be moved about 
the country. 

Only one unit was to be built , and it had to be developed to meet difficult and 
exacting standards. Some of its characteristics include very high vacuum and extremely 
low temperature, simulating conditions in outer space. The optical characteristics 
require high precision with great flexibility to deal with a variety of types of opti- 
cal sensors over a considerable range of the spectrum. For portability, the equipment 
had to be light and rugged, and it was determined that the vacuum tank should be 
designed to meet the ASME code for pressure vessels. Transducer, telemetry and elec- 
tronic equipment to analyze results had to be redesigned and repackaged. 

The quality program for such a one-of-a-kind product of high complexity was a con- 
siderable departure from conventional practice. Because only one unit was to be built, 
we were not overly concerned with the configuration of noncritical hardware. Proof of 
the pudding would be performance in development and checkout testing. For this reason, 
it was concluded that we could dispense with detail parts inspection and depend upon 
go-together as adequate for determining acceptability. If the parts fit, use them. 

If the parts don't fit, make the repair at the easiest point, which might be to modify 
the adjoining hardware. A master print of the assembly drawing was used to record 
such changes, and inspection was used only to verify that such inked corrections accu- 
rately described the hardware change. Only the welded vacuum chamber received detail 
inspection. 

Functional testing was another matter. There was no prototype on which to estab- 
lish performance capabilities and develop acceptance test procedures. The development 
article was the end-item. For this reason we selected a quality engineer to study and 
understand the performance goals, to verify that the planned test actually evaluated 
the required performance, and to monitor conduct of the test by our engineering labor- 
atory team to see that the performance goals were met. It was on this basis that the 
quality engineer was able to certify that the end item met its performance require- 
ments, although the requirements, like the hardware, went through an evolution during 
the course of the program. 

What purpose did QA serve in such a program? Principally it was to keep order in 
a rapidly changing scene. Because of what QA did check, we knew when we were through 
that, sufficiently for the purpose, the hardware was like the marked drawing. We knew 
that the tests measured the performance characteristics correctly. And we knew that 
the actual performance was that represented to the customer. 

LIMITED PRODUCTION IN A HIGH-TECHNOLOGY ITEM 

A different problem is presented by short runs at rather long intervals of a 
high-performance hybrid microcircuit assembly for which reliability requirements are 
extremely challenging. The application is classified, but we can describe the end 
item as the size of a postage stamp, each with eight complex microcircuit chips 
mounted on a multilayer thick-film circuit board assembly and housed in a hermetically 
sealed packaged with thirty wire leads sealed in glass. Test requirements include 
functional electrical tests at two temperatures to the customer's specification at 
each of four points in the overall test cycle. A 100-hour bum-in is performed before 
capping the assembly and again after completing a series of environmental and leak 
tests adapted from MIL-STD-883. Chips are customer furnished and screened at source. 
Butterfly lead frames and the alumina substrates are purchased. We do the rest. 

See Figure 3. 




Figure 3. Hybrid Microcircuit Assembly Before and After Capping 


For the initial order for 60 units , we performed a preplanned inspection of the 
complete manufacturing operation. This included examination of all detail steps. QA 
verified the preparation of screens and the screen printing of both conductors and 
dielectric on three substrate layers . This was followed by continuity and isolation 
testing. Then we verified, step by step, the installation of microcircuit chips, the 
wirebonding, and bonding of the substrate in the alloy box. Precap and final inspec- 
tions completed the full-scale aerospace inspection approach. 

For the second order we concluded that all the verifications just noted could be 
reduced to four examinations by having Manufacturing check off each operation as per- 
formed. QA inspection was limited to examination of the completed substrate, of the 
unlidded assembly before the first precap functional test, of the assembly just 
before capping, and of the fully tested end item before delivery. 

The risk in this change was only with respect to manufacturing cost and schedule 
because of the general visibility in the unlidded box and because of the extensive 
performance testing that followed assembly. In all, nine QA verifications were elimi- 
nated. The acceptance rate in final precap inspection and in test has justified the 
change. A third run two years later has further confirmed the adequacy of control 
with the delegation approach. 

Another change introduced for this product was to design a special-purpose rejec- 
tion tag for use in testing as shown in Figure 4. This consisted of a pre-printed 
failure report permitting immediate entry of exactly what step in the test sequence 
failed. The shop work order shown in Figure 5 was designed to permit stopping at any 
point and recycling. These specialized forms are reproduced for the short runs 
required of this small but sophisticated piece of hardware. A single project-peculiar 
bulletin provides management authority for this departure from the formal material 
review system used for our major programs. 

CONCLUSION 

These are but a few examples of the flexible approach used in Quality Assurance 
to meet the challenge of diversification and of small R&D operations. The specialized 
procedures make optimum use of our standard practices, introducing variance only when 
it saves time and money and still provides control adequate to the purpose of the indi- 
vidual project. The modified systems work in parallel with major space, missile and 
aircraft projects that continue to get the full treatment. They work well because the 
activity is normally confined to a limited work area and performed by small teams who 
can be quickly instructed in special procedures for special cases. 
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Figure 5. Specialized Shop Work Order 
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COST OF QUALITY IN THE SERVICE INDUSTRIES 


George Rosenzweig, Vice-President 
Technitrol Testing Laboratories Ltd. 
Mississauga | Ontario. 


INTRODUCTION 

A review of the Quality Cost Literature based on the extensive 
index compiled by the ASQC Quality Costs Committee, revealed, that 
out of the 213 titles listed, 209 papers deal mainly or entirely with 
manufacturing applications, while the remaining four papers are 
related to the cost of clerical errors and purchased hardware quality, 
in the service industries. 

This gross neglect of the topic of my title must have some well 
based reason. The search for this reason prompted me to conduct this 
exploratory study that intends to probe the viability of the Manufact- 
uring Quality Costs Systems within the Service Industries. 

The growing importance and size of the Service Industries should 
also be taken into consideration, particularly now, when in North 
America there are more people employed by Service Industries than by 
Manufacturing and Other Industries combined. 

STUDY - STAGE 1 

As a first step I held some informal discussions with Executives 
of services of a hospital, a Hotel and some Government Agencies. These 
discussions helped me to formulate the following observations. 

Basic Quality Control terminology that is common in the Manufac- 
turing Environment is not understood in the Service Industries. 
Questions that we often ask to evaluate Quality Systems may sound 
quite strange when put into the service environment. 

Let us only mention a few: 

"What is the Reporting Relationship of your Quality Manager 
within the Organization?" 

"Do you have an up-to-date Quality Manual?" 

"Do you have an active Quality Program?" 

w Do you have a Quality Cost System?" 

"How is your Reliability and Maintainability Program fit into 
the Quality Organizations?" 

It would be erroneous to conclude at this point that the Service 
Industries examined are not adequately organized. Nevertheless, it 
is obvious that they are differently organized, and the single central- 
ized Quality Function customary in the Manufacturing Industry titled 
as the Quality Control or Quality Assurance Manager has not emerged 
yet. 


The design and control of quality in Hospitals, Hotels, Offices, 
etc... is inter-woven in their operating system, without one specific 
Executive being responsible for its co-ordination, and without clear 
definitions. 

Discovery Number 1 - Management 

^Service Industries are rarely structured for Vertical 
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Quality Organization 


The different aspects of quality are the responsibilities of 
several different Departments and Managers under most diverse titles 
without a central monitoring authority* This person is usually other 
than the Top Executive. 

Some typical titles for Service Industry Quality Bosses are named 
here: Head Chef, Chief Housekeeper, Bell Captain, Head Nurse, Public 
Relations Director, Director of Pharmaceutical Services, Credit Manager 
Catering Manager; It is easy to notice that these Managers are also 
responsible for the performance of the service itself. 

D iscovery Number 11 - Standards 

^Standards of Services provided are not always defined from the 
Fitness for Use point of view. While the tolerances are very rigid 
or non existent, their enforcement and interpretation is very loose." 

It is further complicated by Quality parameters such as timeliness 
politeness and manners. Operating Procedures manuals are inventory and 
finance oriented. Acceptance or Rejection Criteria are rarely stated 
in writing. 

Discovery Number 111 - Measurement of Conformance 

"Measurement of 1 Quality within the Service Industries, does not 
follow the traditions of manufacturing scoreboards." 

The concept of failure is mostly a maintenance problem, and 
usually the total service provided only suffers a degredation rather 
than an outright failure. 

For example, what is a "Failure" in a restaurant? When a 
customer fails to return a second time, dies of food poisoning, rejects 
the food, or the waiter drops it on the floor? - or a combination of 
several characteristics? 

As service industries are heavily dependent on return business, 
the criticality of non catastrophic failures should be particularly 
emphasised. 

At this stage of the study I had to return to the original title 
of the paper and draw an interim conclusion. For the Quality Profess- 
ional, that emerged from the Manufacturing Industry, the immediate 
opportunities to apply his concept of Quality Costs is limited or 
remote. 

It is common to try to apply a proven methodology in a new 
situation, only to realize that the differences outnumber the common- 
alities. 

What is then the source of the incompatibility? 

In my opinion the basic cause behind this observation stems from 
the fact that Quality Costs Systems are second generation Quality 
Control tools, that are successfully applied only after an already 
existing first generation Quality System is in place. 

This observation may be considered as a personal opinion, and by 
no means indicates that the writer surrendered to the Quality Control 
Practitioners* arch enemy; — "This will not work here" — . 

To put this into a positive light, we should say the basic 
pre-requisites are not in place therefore the system will not function 
here. 

To overcome this difficulty further studies are required. 



STUDY - STAGE 2 


The second stage of the study was then launched to explore 
alternate but complimentary avenues. 

a) A Systems Engineering Survey was conducted, with the 
objective of drawing up a map of all activities that have high 
content of Quality Work Elements. This survey also included 
the responsibilities and relevant available reporting and 
recording activities. As these studies were conducted for 
academic purposes, no actual action was taken on the findings, 
at least not for the time being. 

The only benefits at this point were the better understanding 
of controllable variables. 

b) A study for discovery of opportunities for errors, material 
losses and customer dissatisfaction was then conducted. This 
study was mainly a review of the service-s provided and the 
activities required to perform the service-s . 

The results were quite surprising. There were probably more 
opportunities for errors and loss of business, discovered than 
originally anticipated. A further disturbing fact came to light? 
the recognition that the order of magnitude is unknown and trends are 
not depicted. 

The following observations are not intended to criticize the 
operations of the institution under study, only to highlight the 
elements of Quality Opportunities for control and improvement. 

Hospital - Survey 

Each functional director is responsible for the activity within 
its jurisdiction. There are no appointed individuals performing 
Inspection, Quality Control or Audit Functions and there is no date 
generated for analysis and corrective actions. 

Quality losses or quality costs are generated within each 
functional jurisdiction, but none of them identifies them as such, 
however they include them in the budget as traditional facts of life; 

-Medication delivery not in accordance with Doctor* s instructions 
-Medical instrumentation not fully serviceable 
-Pilferage and breakage of equipment 

-Availability of Medical and Housekeeping staff not co-ordinated 
with patient population 

-Non availability of accommodation when patients are admitted 
-Errors in Quantities and Qualities of Food supplies 
-Lack of syncronization of housekeeping with patient care 
-Building and Maintenance failures, reliability and maintain- 
ability problems 

-Record errors, storage and retrieval 

-Clerical errors in accounts receivables and accounts payable 
-Unnecessary laboratory work, due to administrative and 
identification errors 
-Liability claims 

-Moral and attitude problems of the employees 

The need for scorekeeping on degree of compliance with standards 
is obvious. If we had the; who should do what, when and how, and had 
someone record, the deviations and losses within the different function 
sal directorates (overlapping geography) we could easily add the dollar 
values. The operating instructions are more implied than specified. 

At present, instead of Quality Cost Reduction programs, most 
Service Institutes are budgeting for traditional "losses** as part 
of the Historical Operating Cost. Maybe we as taxpaying citizens 
should challenge the "Unknown Losses" because they may not be 



"Unavoidable Losses", It is not uncommon to hear budgeting for 10$ 
food spoilage, 5$ linen pilferage and a very extensive list of medical 
supply disposal due to expired shelf life. 

The question of avoidable losses and the philosophy of "planning 
for errors" is a serious problem to contend with. 

Hotel - Survey 

The similarities in organization and sources of unqualities 
between a Hospital and a Hotel were striking. 

The concepts for Quality Control are almost identical; 

-Services delivered not in accordance with customers request 
or expectations 
-Reservation system errors 

-Availability of staff not syncronized with public needs 
-Errors in quantities and qualities of food supplies 
-Failures in Electro/Mechanical equipment and long response 
time for correction 

-Timing of housekeeping in conflict with guests* comfort 
-Clerical errors in billing and accounts payable 
-Unnecessary services offered and rendered 
-Pilferage by staff and guests 
-Liability claims 

-Moral and attitude problems of the employees 

Due to the fact that the quality losses are directly effecting 
the profitability of the operation, we may find an earlier breakthrough 
in this field. Nevertheless, the groundwork is still missing. 


SUMMARY 

An attempt to apply Quality Cost Systems to the Service 
Industries at the present time and in state of the art would be rather 
premature • 

To hang price tags on items that are not fully defined for the 
majority of Service Industries, and to attempt to transplant manu- 
facturing titles and labels with different meanings is an invitation 
for failure or at least for confusion. 

It seems that an enormous opportunity and probably an equally 
large need exists to develop the First Generation Quality Systems for 
the Service Industries. 

It will require the identification and recognition of the 
universal aspects of quality within the different Service Industries, 
minimizing the specifics, emphasizing the commonalities. 

The experiences learned by the Quality Practitioners in the 
Manufacturing Industries could become very useful, we must always 
bear in mind, that Management and People Problems are more difficult 
to deal with than the technical problems. Therefore the common ground 
at least at this point is in existance. 

When this First Generation Quality System Development will reach 
its maturity, the methodology of Quality Costs Systems will become 
timely. It may require major changes from the present manufacturing 
format to the future Service Industry model. 

As executives of all Industries are most familiar with dollar 
and cent reports, the arrival of the Quality Cost System will become 
beneficial and well received but not before the quality system itself 
is established and well understood. 

The concepts of Total Quality Control may very well be the 
approach but will have to be re-developed from first principles 
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. . as a effort of the Quality Professional and the 

expert of the specific Service Industry involved* 
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THE ROLE OF QUALITY IN BANKING 


Thurmon Sutcliffe, Manager 

Quality Assurance & Quality Control 

Chemical Bank - New York 

55 Water Street 

New York, New York 10041 


To effectively demonstrate the role of Quality as it relates to 
Banking, we must first analyze our objectives. 

In Banking, we have a two fold objective — customer satisfaction 
and the availability of funds. 

To attain these goal objectives, we must design and implement 
programs and measurements that will permit us to fine tune the system 
for maximum efficiency. 

First, we must through statistical applications determine the 
bandwidth of performance for our clerical staff, as well as our hard- 
ware and the product we are processing. 

Reductions in defectives (rejects) will give us increased avail- 
ability of funds and customer satisfaction. 

To establish a Quality Assurance and Quality Control System to 
attain these objectives is, to say the least, an extremely difficult 
task, but we at Chemical Bank have made tremendous progress in this 
direction and we would like to share our story with each of you. 

Our first recommendation is a Pre-delivery Sampling Program of 
all customer checks, prior to customer delivery. This requires a 
cooperative effort by the Purchasing Department, account officers, 
and an active role by the branch managers. A sampling based on an 
AQL of 0.4% will give results of a very substantial payback. 

This program of on-line testing of checks will verify compliance 
with ABA specifications pertaining to MICR (Magnetic Ink Character 
Recognition) , routing transit number and account number verification. 
A feedback mechanism is then established with Purchasing and the 
check printers developing measurement standards on submittals and a 
corrective action program that will enhance production efficiencies 
on the part of the check printers and improve the quality of the 
checks . 

Design is an important ingredient relative to the product and 
has a great impact on over-the-counter deposit tickets. Debit memos 
and credit memos — from the standpoint of size, paper weight, and 
legibility of information — are an important ingredient to accomplish 
the task of improved quality in this area. It is important for 
Quality Control to be an integral part of the design review and sign- 
off process. 

As in every production process our human resources become our 
most important element for success or failure in the attainment of 
our goal objectives. 

We at ChemBank have instituted Quality Sampling Programs to 
monitor the performance of our operators by designing forms and 
itemizing possible errors that an operator can make; we then estab- 
lish a random table selection for operator sampling. Weekly charts 
are submitted and by designing an error/operator matrix, error 
patterns can be analyzed for immediate corrective action. This 
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feedback of data including weekly process average and process 
capability reporting , determines which operators are to be monitored 
for defined error categories. 

Last# but most significant/ is the monitoring of hardware per- 
formance through software applications and statistical applications; 
goal objectives are determined for encoder and sorter performance. 

No longer is a subjective judgement of projected performance 
accepted. All "Super Goal" objectives are defined through sound 
statistical applications. 

Once these programs are in place, we can validate our sampling 
programs to our actual performance, thereby tailoring our operating 
characteristic curves to tighten or reduce our sampling based upon 
our actual performance. 

By developing a "Monthly Performance Report on Quality Indica- 
tors," we can obtain an analysis of encoder performance, sorter 
performance, and measurements of product lines, i.e., Incoming, Mail, 
P & R, Total Volume, and "On-Us" items. 

As a follow-up, measurements are made on errors impacting 
customer statements, determining cause and correction prior to 
statement rendering. 

This is followed by postal card surveys to customers on a ran- 
dom basis for any statement discrepancies or other comments; as 
customer satisfaction is our prime objective. 
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MICROCOMPUTERS AND SOFTWARE QUALITY CONTROL 


Roger W. Berger, Associate Professor 
Department of Industrial Engineering, Iowa State University 
Ames, Iowa 


INTRODUCTION 

Microprocessors - - you can hold a dozen in one hand, yet each may have more raw 
computing power than the largest computer which existed in 1949. Such is the rate of 
change in the computer industry. The microprocessor was born in 1971 when an Intel 
engineer, M. E. Hoff, Jr., developed an integrated circuit (IC) chip for a Japanese 
calculator company, Busicom. Busicom wanted a family of business calculator products 
to meet various market requirements. Hoff felt that to custom design a whole series 
of chips would not be very cost-effective, and proposed an elegant solution: a cen- 
tral processing unit (CPU) chip would be designed to interact with a read-only memory 
(RAM) chip, to form a stored-program computer. Then each specific calculator would 
be implemented by a particular program stored in ROM. 

The job of customizing calculators was thus transformed from building different 
hardware to writing different software, a major intellectual breakthrough. The CPU 
Hoff designed was named the Intel 4004. It used a 4 bit data structure and p-channel 
metal oxide semiconductor (PMOS) technology. Although the 4004 was rather slow, and 
did not originally attract any attention in the computer industry, it started a revo- 
lution the end of which is nowhere in sight. Intel followed it with the 4040, the 
8008 (using 8 bits), the 8080 (using n-channel technology or NM0S), the 8080A (cur- 
rently an industry standard) the 8048, and the 8085. Each successive chip tended to 
run faster, provide more functions, use less power, and cost less money than its pre- 
decessor. 

Soon competition appeared and the industry was born. Motorola, Rockwell, Texas 
Instruments, National Semiconductor, and other large firms entered the microprocessor 
market. Small companies such as M0S Technology, M0STEK and Zilog developed new chip 
designs and provided second sources for established ones. Today about 60 companies 
are manufacturing microprocessors, and thousands of companies are building them into 
other products. 


MICROCOMPUTER RELATED PRODUCTS 

Since there are several microcomputer-related products which are often confused, 
some definitions are in order. With the passage of time, the more innovative manu- 
facturers of each product type have incorporated some of the better features of other 
types into their products, so the distinctions will become more blurred as the indus- 
try evolves. These definitions and examples represent each type as originally con- 
cei ved. 

Microprocessors 

A microcrocessor is a large scale integrated (LSI) digital processing circuit on 
a single silicon chip. It contains registers, data paths, and control logic which 
permits the execution of programs. The machine language instruction set is completely 
defined by the control logic. Prices may range from $3.00 to $50.00. The chip may 
be mounted on a dual -inline package (DIP) providing the gourmet computerist with chip- 
on-DIP. A microprocessor does not include any access or display methods such as key- 
board or 1 ight-emitting-diodes (LEDs) , or nor does it include any memory for storing 
programs or data. Microprocessors are built into many products besides computers, 
such as machine tools, auto ignitions, calculators, home appliances, and games. Rep- 
resentative products are the 8080A, the Motorola 6800, the Zilog Z-80, and Texas In- 
struments 990. 
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Microcomputers 


A microcomputer is a module which consists of a microprocessor, some memory, nor- 
mally both RAM and ROM, input-output interface circuits, a clock, and other logic such 
as buffers and drivers. The earlier microcomputers placed each of these functions on a 
separate chip, then hooked them together on one or more printed circuit boards. How- 
ever, new microcomputers such as the Intel 8048 include most or all functions on one 
chip, giving rise to the expression, "computer on a chip." Some popular microcompu- 
ters are the Intel SBC80/10, the Altair 8800, the Heathkit H-ll, and National Semi- 
conductor's SC/MP. Prices run between $100 and $1200. 

Microcomputer Systems 

A microcomputer system is created by attaching various input-output and auxiliary 
storage devices to a microcomputer. There are many subgroups in this product clas- 
sification, with new ones appearing all the time. One class is the microcomputer 
development system (MDS) which can be used to develop new microcomputer systems as 
well as write programs in assembly language and high-level language. A leading ex- 
ample is Intel's INTELLEC system, which retails for about $20,000. In contrast is 
the home computer, which includes a keyboard, CRT, and cassette tape unit. The two 
most popular home computers are presently the Commodore PET 2001 and the Radio Shack 
TRS-80. Both were introduced in late 1977, retail for $600, and include a BASIC lan- 
guage processor in ROM. In between the two above extremes are small business systems, 
laboratory data systems, and hobbyist systems. By thumbing through a recent issue of 
Byte or Mini -Micro Systems one can see many examples. Some brand names include Poly- 
morphic, IMSAI, Processor Technology, Apple, Heath and Digital Group. 

Mini computers 

These have a much longer heritage than microcomputers, having first burst upon 
the scene in the early '60' s. The original distinctions were along the lines of price 
(minis about 5 times as expensive), speed (minis operating at about three times as 
fast) and word size (typical minis were 16 bits, vs. 8 bits for the micros). The 
company largely responsible for the minicomputer industry is Digital Equipment Cor- 
poration, and its most successful products have been the PDP-8 and the PDP-11. The 
latter design was converted into integrated circuit technology and renamed the 
LSI-11, which then also became the H-ll under a licensing agreement with Heath. This 
switch left the architecture unchanged, but reduced the price, speed, power require- 
ments, and volume. So is the H-ll a minicomputer or a microcomputer? Other leading 
minicomputer manufacturers are Data General, Computer Automation, Hewlett-Packard, and 
Wang. 

Pro grammable Pocket Calculators 

These are - sometimes confused with microcomputers. This is understandable, since 
there is a microprocessor in every programmable calculator. The important distinction 
is that the capabilities of the calculator are restricted to numerical calculations 
using a sealed control program; whereas a true computer can be loaded with any type 
program and data the user sees fit. The programmable calculators, though less flex- 
ible, are much easier to use than microcomputers for straight numerical computations. 
In fact, numerical computations are distinctly difficult for microcomputers. Either 
a separate chip or special software is required to do this. The Hewlett-Packard 
HP-65 was the first programmable pocket calculator. It sold for $800.00. A currently 
popular programmable calculator is the SR-59, sold by Texas Instruments for about 
$300.00. This calculator has a printer and magnetic card storage. It also has pro- 
gram libraries stored in removable memory modules, which provide for extremely con- 
venient program execution. 


MICROCOMPUTER SOFTWARE 

When microcomputers first became available, software development facilities did 
not exist. System engineers were forced to translate programs into machine code by 
hand and then toggle them into the machine through the front panel. This state of 
affairs has now improved, but the facilities for debugging and developing microcom- 
puter programs are still far more primitive than those available to large computers. 



Cl asses of Software 


For purposes of this discussion, microcomputer software is divided into three 
classes: (1) systems, (2) developmental, and (3) applications. Systems software, is 
that which helps a fully developed system to provide the service for which it was 
originally acquired. The operating system is the prime example. Developmental soft- 
ware includes language processors to create software, such as assemblers, BASIC, 
PASCAL, PL/M and muPro; and test software to validate programs, such as the National 
Bureau of Standards test series (1). Applications software constitutes all the pro- 
grams which perform the final end result of the computer. Text editors, mathematical 
routines, computer game programs, and machine control software are examples of the 
wide spectrum of applications programs. There are currently many thousands of such 
application programs in existence, and more are being created every day. They are 
being sold, licensed, exchanged in User Groups, and given away. Little is known of 
their quality or reliability! 

How Much Software Do Micros Require? 

Depending on the use planned for the computer, a whole lot or very little soft- 
ware might be required. Computers which are imbedded into products will normally have 
one program written to perform the desired function, which is then permanently stored 
into a read only memory (ROM). (Such a program is often called firmware, gray area 
between software and hardware.) On the other hand, a general purpose machine such as 
the MDS or a small business system, must use a wide variety of software to justify its 
purchase. Various language processors and application programs may be stored on 
floppy disks or cassette tapes. Depending on the resources of the organization, the 
programmer, developer, and user of microcomputer systems may have to accept rather 
primitive software for testing and debugging. This is one major challenge for micro- 
computer software quality control. Large system programmers are used to elaborate 
diagnostics, tracebacks, and dumps. These software aids have evolved over the years 
are are in large measure dependent on the large memories available. No such sophis- 
ticated developmental software exists (as yet) for the microcomputer programmer. This 
means a much higher level of training, planning, self-disciplined step-by-step check- 
out by the programmer is required. Quality control people can provide a real service 
to the profession and the industry by participating in such a program. 


RELIABILITY OF MICROCOMPUTER SOFTWARE 

A computer user supposedly asked the manufacturer's representative why he had 
so much more trouble getting the software to work than the hardware. "Why are your 
engineers smarter than your programmers? Why don't you test your software as well 
as your hardware" ( 2). The answer is that software is not only inherently more com- 
plex, it is fundamentally different from hardware. A purely intellectual product, its 
success or failure depends on many external factors which are not easily controlled 
or tested. 

Software Errors 


Myers (2 ) defines a software error as an incident where the software does not 
do what the user reasonably expects it to. This definition puts the burden on the 
programmer to understand what the user may reasonably expect; a software error is not 
a property solely of the software. Careful definitions of requirements and program 
restrictions are essential. 

Cost of Software Errors 


Foster (3 ) estimated that software costs from ten to three hundred dollars per 
instruction. To most programmers this seems unreasonable high, but when one recog- 
nizes the total spectrum of activity which must be charged to each instruction it is 
not quite so unreasonable. For instance, Myers (2 ) claims that program coding ac- 
counts for only about 10% of the total cost of software, the other 90% being in de- 
sign, testing, and maintenance. 

If a statement costs from $10.00 to $300.00, what does a "bug" or software error 
cost? Like most defects, it depends on where the bug occurs. A missing comma in a 
FORTRAN program in an early space launch caused the destruction of a multimillion 
dollar rocket. This is a spectacular but exceptional case. As another example, if a 
product is manufactured with a bug which must be later removed by replacing the ROM, 



the cost is very possibly going to be in the order of $100.00 for parts and labor on 
each copy of the product which has reached the field. If several months have passed 
since product marketing, over 1000 copies would probably exist. So one bug in the 
wrong place could easily cost over $100,000. And this would far exceed the total 
development cost in most cases. The mass production of microcomputers creates quality 
cost problems never encountered in the larger systems. Microcomputer software relia- 
bility is important! 


ACHIEVING SOFTWARE RELIABILITY 

Reliability is one of several conflicting objectives of software development. 
Cost, delivery time, and function are also important parameters. The ideal software 
which is extremely powerful, highly reliable, low in cost, and quickly produced, 
clearly does not exist. Thus tradeoffs must be made, based on perceptions of market 
requirements and the organization's capabilities, just as J. M. Juran has explained 
to us for many years. To summarize much that has been published in symposia, pro- 
ceedings, books, and technical journals, we offer the following fundamentals: 

1. Provide tools and skills appropriate to the task. A wide array of develop- 
ment systems, software test programs, software analysis systems, etc., are available. 
Numerous short courses are being offered on structured programming and software en- 
gineering. There will always be a shortage of highly qualified programmers who write 
error free code the first time! 

2. Apply sound program design techniques. A careful requirements analysis, un- 
ambiguous specifications, appropriate documentation, and structured programming are 
examples. 


3. Test software thoroughly. The importance of adequate testing can hardly be 
overemphasized, and is discussed in further detail below. 

4. Control change once the program leaves the hands of the oriqinal programmer. 
Foster (3 ) Provides many details regarding change controls and documentation. 

Software Testing 

The Association for Computing Machinery (ACM) sponsored a program testing sym- 
posium in 1973 (4 ) which provided the basis for the following definitions, festi ng 
is the process of program execution with the goal of finding errors. Verification 
is testing in a simulated environment. Val idation is testing in a real environment. 
Other important terms are module testing, integration testing, system testing, ac- 
ceptance testing, and installation testing. These terms have obvious analogues in 
traditional manufacturing quality control. 
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It is rather generally agreed that no programmer should be expected to test his 
own program. There appear to be two fundamentally opposing philosophies as to how to 
involve others in the testing. One is the structured walk-through of the Chief Pro- 
grammer Team, where colleagues of the programmer study the program carefully and at- 
tempt to help the programmer improve it. The other is the independent verification 
process managed by a separate organizational group. A summary of the development and 
test process is given in Table I, which was adapted from (5). 

A study of Table I shows that for each product created in the software development 
process, one task is performed by the design group and an analogous one is performed 
by the verification group. This type of approach is probably not common for micro 
software. 

More common is the approach taken by Insite, the Intel Users' Program Library. 
Insite is one of many sofware libraries growing rapidly to support the microcomputer 
movement. They are being developed both by computer manufacturers and by independent 
software houses. They tend to emphasize application programs. Programs submitted to 
Insite will support all computers based upon the 8080, 8085, and 8048, and many of them 
would with monor modifications support somputers based on the Z-80. There are hundreds 
of programs, in fields varying from engineering analysis to statistics to games and 
peripheral control. Members are encouraged to submit programs, which when accepted 
earn a year's free membership, or five free paper tapes, or one free program diskette. 
Any of these options is worth approximately $10. 

Now it is obvious that for $10 in service a submitter does not have an incentive 
to do a great deal of program testing. Insite does not in any way guarantee the pro- 
grams, or attempt to test them. It does require that in addition to source listing, 
source code on paper tape or diskette, and the usual factual details, that the program- 
mer provide "a test program which assures the validity of the contributed program." 

The exact nature of the test program is not specified beyond the quoted phrase. One 
would assume that Insite expects the programmer to choose some representative data, run 
the program with this data, and submit the successful run as the test program. What 
kind of validation is that? What is the probaiblity that the software obtained will 
perform in the way the user reasonable expects it to? Neither Insite nor any of the 
other software libraries provide a quantitative answer to that question, but the reader 
can guess that the probability is low. 

It is not intended to single out Intel for criticism. In fact, Intel is one of the 
most responsible companies in the microcomputer business, and the price being paid for 
the software cannot support a great deal of testing. Nevertheless, many unsophisticat- 
ed buyers of microcomputer software are buying, and will continue to be buying, soft- 
ware with bugs in it. When these bugs spring forth, many of the users will be comple- 
tely baffled as to whether they have a user error, a software. Many will lack both 
the skills and the equipment to make the determination. 

Some sort of national effort is needed to monitor the quality of microcomputer 
software, and to provide objective standards against which each source of microcomputer 
software can be judged. In other consumer items there are organizations like Consumer's 
Union. Where insurance hazards exist, there is the Underwriter' s Laboratory. Who will 
perform this service for the microcomputer software? The National Bureau of Standards? 
The Associate for Computing Machinery? The American Society for Quality Control? 
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A PROGRAM FOR SOFTWARE QUALITY ASSURANCE 


Kurt F. Fischer 

TRW Defense and Space Systems Group 
Redondo Beach, California 90278 


INTRODUCTION 

At TRW Defense and Space Systems Group the need for a division-wide organization to assist software 
management became quite apparent during the late 1960’s. Like most other software vendors, TRW was 
concerned about the frequent cost overruns and schedule slippages in its software projects, and decided to 
develop methods to turn this trend around. As part of that decision, it established its first software quality 
assurance staff in 1969. Though small in number, this group was staffed by very strong management and 
technical people. The first few months were spent doing what many new organizations do: defining their 
job, hiring their staff, and marketing their services. Much of the early effort was spent writing QA policies, 
plans, and procedures, developing automated QA tools, and participating on proposal teams. This effort 
paid-off handsomely as the software subcontract for the Site Defense Program (later known as the BMD 
Systems Technology Program) was proposed in 1971 and won in 1972 with a very heavy (40 man year) 
dedicated quality assurance role. 

Managers in other projects soon saw that the software quality assurance function could solve many of 
their problems, and slowly they began to employ QA personnel both in a full time project-dedicated capacity, 
and on an as needed basis. As software quality assurance became more active in a wider variety of projects 
and environments, the tasks it performed became better defined and more standardized among the projects in 
which they were employed. The remainder of this paper describes more fully the purpose, functions, organi- 
zation, benefits, and lessons learned with regard to the current implementation of software QA at TRW. 

PURPOSE 

The purpose of software quality assurance is clearly to assure the quality of software products. However, 
the words “assure” and “quality” can be interpreted in several ways, so it is important to begin with a dis- 
cussion of the role of quality assurance (QA). The purpose of software QA as stated in MIL-S-52779 (Soft- 
ware QA Program Requirements) is “. . . to assure that software delivered under the contract complies with 
the requirements of the contract.” The problem with this definition is that if a contract specifies poor soft- 
ware quality (inadequate documentation, poor standards, insufficient testing, etc.) the software QA program 
will assure that you get poor quality software. An expanded definition of the purpose of software quality 
assurance is: 

To assure sufficient planning and control to affect the development of software products 
which meet the specified and intended requirements in the areas of: 

• Accountability 

• Usability 

• Testability 

• Reliability 

• Maintainability 


FUNCTIONS 

The detailed functions of software QA vary from one project to another, but generally can be grouped 
into 8 functions: 

1 . Initial quality planning 

2. Development of software standards and procedures 

3. Development of quality assurance tools 

4. Conduct of QA reviews and audits 

5. Inspection and surveillance of formal tests 

6. Configuration verifications 
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7. Management of the discrepancy reporting system 

8. Retention of QA records. 

Initial Quality Planning 

Experience has shown that like other project functions, successful implementation of the QA program 
depends heavily on QA planning during the early phases of the software development life cycle. This task is 
accomplished by a complete review of early project documentation (e.g., the Contract Statement of Work, 
System Specifications, Project Plan, etc.) The review culminates in the preparation of a QA Plan which con- 
tains the quality assurance functions, tasks, responsibilities, and identifies the QA tools needed to assure 
sufficient software quality with regard to accountability, testability, usability, maintainability, and reli- 
ability. The QA plan is then reviewed by other project organizations, and approved by the program manager 
and customer. After QA Plan approval, task assignments are made to carry out QA functions, however, these 
assignments are usually based on level-of-effort and must remain flexible to adapt to: 

• The needs of the current phase in the development life cycle 

• Shifts in attention-needing areas (e.g., technical problems) 

• Unexpected demands placed on QA by the project manager 

After QA Plan approval,. QA policies and procedures are written which describe the methods and 
procedures to be used in implementing the quality assurance requirements as defined in the: 

• System Specification 

• Contract Statement of Work 

• Project Plan 

• QA Plan 

A sample table of contents for QA policies and their associated procedures is shown in Table 1. 

Table 1 . Sample QA Policies and Procedures Titles 


1. QA Management 

2. Reviews and Audits 

2.1 Specification Review 

2.2 QA Audits 

2.3 Standards Compliance 

2.4 Unit Development Audits 

2.5 Test Plan and Procedure Review 

3. Inspection and Test Surveillance 

3.1 Unit Test 

3.2 Integration Test 

3.3 Validation Test 

3.4 Test Review Board Operations 


4. Software Turnover 

4. 1 Software Installation 

5. Discrepancy Reporting 

5.1 Discrepancy Analysis 

5.2 Discrepancy Surveillance 

5.3 Deviations and Waivers 

6. Configuration Verification 

6. 1 Configuration Verification 

7. Quality Assurance Records 

7. 1 Description of QA Records 

7.2 Discrepancy Report File Maintenance 


The payoffs of continually planning, setting priorities, and maintaining project-wide visibility cannot be 
underestimated. QA effectiveness is directly proportional to the visibility that QA has into the project and 
the organization’s involvement in the planning process. 

Development of Software Standards 

The purpose of software standards is to improve the maintainability and readability of the software 
product. At TRW, a comprehensive and detailed software standards program has been successfully insti- 
tuted to achieve this purpose. This success can be largely attributed to two factors: 

1. Software standards are not dictated from the executive offices, the project offices, or 
from Quality Assurance, but are developed out of close communication among the 
design and development, test, QA, and project offices. 



2. A tool has been provided to automatically check the software against most of the 
standards. This allows programmers to audit themselves so that there are no 
surprises at turnover time. 

Waivers and deviations complement the standards by providing mechanisms for handling allowable non- 
conformances with the standards. They provide permanent or temporary relief, respectively, from standards 
compliance due to, for example, technical difficulties, inefficiencies, or schedule impact, and must be 
approved by the QA and project managers. 

Development of QA Tools 

The short history of software development is laden with excuses for poor performance. True, many of 
the excuses are due to people problems, but many also are due to technical problems. While tools can’t solve 
the people problems (unless the tool replaces the people), they have been used with great success to solve many 
problems in technical areas. For example, tools perform many functions in the areas of file management, 
data base generation, code verification, and test assistance. Generally software tools should perform tasks 
that are more cost effective and accurate if done automatically than if done manually. This includes tasks 
that are often tedious, menial, boring, difficult, error prone, repetitive, and costly. Therefore it is not 
surprising that quality assurance groups make heavy use of software tools. 

QA tools are essential to the successful implementation of a software QA program because they provide 
a cost effective method for accurately evaluating large software products in a short period of time. At TRW, 
software QA personnel have developed numerous tools to assist in their tasks, but have found the below 
sample perhaps the most highly successful. 

Product Assurance Confidence Evaluator (PACE) — PACE is a unique tool to aid program developers and 
testers in the planning, execution, and evaluation of both unit level and program level tests. The object of 
PACE is to quantitatively assess how thoroughly and rigorously a program has been tested. PACE is the tool 
that is used to assure that every instruction and branch be subjected to an execution test. 

Fortran Code Auditor -- The Fortran Code Auditor is the first known automatic standards auditor in the 
software industry. It was developed in 1973 to audit eighteen software standards, and has increased today to 
thirty-four software standards. Code Auditor is heavily used by both programmers and quality assurance 
engineers. It has been a valuable tool because it permits the enforcement of a comprehensive set of coding 
standards which reduces the number of software errors, improves execution efficiency, and improves read- 
ability and maintainability. 

The benefit of the Fortran Code Auditors is that it not only automatically audits a large amount of code in a 
matter of seconds, but provides a means for programmers to audit themselves prior to turnover. 

Structured Programming Auflitor (STRUCT) - STRUCT is the QA tool used to verify compliance with the 
structured programming standard. It is executed after PACE, as it uses the Segment Transfer Table as output 
by PACE to convert each source routine into its directed graph representation. STRUCT then performs a 
flowgraph reduction algorithm on the Segment Transfer Table to determine if each routine is structured 
according to the six following constructs: sequence, if-then-else, do-while, do-until, case, escape-from-loop. 

Units Consistency Analysis (UCA) - The UCA Program scans Fortran source code and its associated data 
base (for access to variable units definition) and symbolically interprets the equations referencing the variables 
to assure that the units are correct and consistent within the assignment statements. For example, in the 
assignment statement 

C = A * B 

if variables A and B were defined to be in units of feet, variable C ought to be defined in units of square feet 
or a diagnostic would be printed. 

Conduct of QA Reviews and Audits 

Tie most important task for QA is to ensure that the QA policies, procedures, and software standards 
developed and identified in the appropriate documents are carried out. To this end, informal and formal QA 
reviews and audits are conducted incrementally on software products. 


In the context in which they are employed at TRW, reviews are QA critiques performed against 
documents, whereas audits are QA critiques performed against processes, often scheduled near a milestone 
event in that process. The criteria against which documents are reviewed are: 

• Adherence to format and pagination standards 

• Clarity of objectives 

• Technical content 

• Interdocument consistency 

• Traceability to higher level specifications 

QA audits usually include document reviews, and in general perform four functions: 

• They assess compliance of source code and documentation to software standards and 
procedures; 

• They assure traceability of requirements 

• They determine the satisfaction of system requirements during system test and 
acceptance phase; 

• They assess test sufficiency. 

The following audits may be performed on software projects: 

Software Requirements Audit - This audit is conducted prior to the system design review and includes the 
review of the software specifications, projects plan, and test specifications. QA reviews these documents to 
determine whether: 

1 , Software requirements are traceable to system requirements 

2. The planned testing will assure satisfaction of software requirements. 

Interface Verification Audit - The interface requirements, design, and programming specifications are 
audited as early as possible to identify and correct potential interface problems. The process of auditing an 
interface specification is similar to the above. QA personnel may also participate in a weekly meeting of the 
interface control working group. 

Preliminary and Detailed Design Audits - These audits are conducted prior to PDR and CDR, respectively, 
and are concerned with the format and content of the design documentation and test plans. Results of the 
findings are discussed during the PDR and CDR, respectively. 

Incremental Development Audit — The incremental development audit approach involves the conduct of 
periodic audits during the code and unit test phase. This allows the developers time to implement corrective 
actions in their Unit Development Folders (internal programmer documentation) and code without seriously 
affecting cost and schedule performance. The use of checklists facilitates the audit, as it standardizes the 
audit process and lets the developers know ahead of time what to expect. Incremental development audits 
include: 

1. Verifying that changes to requirements, design, and interface definitions are 
maintained and updated; 

2. Verifying that any applicable design or code walk-throughs were properly 
held; 

3. Audit of source code for standards compliance; 

4. Assessment of test sufficiency or thoroughness of unit testing; 

5. Adequacy of configuration identification and control. 

Pre-Turnover Audit - Prior to the turnover (delivery) of the software to the test team, a pre-turnover audit 
is conducted to assess the adequacy of unit testing. QA evaluates the total software product being turned 
over in terms of : 

• Actual versus expected test output 

• Change control status verification of code 

• Completeness, content, and organization of Unit Development Folders 

• Code compliance to applicable software standards 

• Data base control 


• Test sufficiency 

• Discrepancy report status 


This audit must be well planned and executed efficiently in order to obviate schedule impacts. Adequate 
time should be allotted to negotiate corrective action items and correct deficiencies in the software documen- 
tation prior to deliver to the test team. This audit is complete when QA certifies the condition of the soft- 
ware, and the test manager accepts the turnover package. 


Test Audit - 


Software test audits are conducted at the completion of each test phase. Their purpose is to: 


• Assure that sufficient software configuration management procedures are 
being followed; 

• Assure that test specifications used by the test group are the current 
approved versions; 

• Assure that test reports identify proper test procedures and software 
configuration , specify the test analysis, and if any deficiencies or 
deviations were noted, how they were explained and accounted for; 

• Verify that test procedures provide a step-by-step rationale for conducting 
a test, and that test results comply with acceptance criteria specified in 
the test procedure; 

• Verify that test data packages comply with approved formats. A test data 
package is a stand-alone set of documents which contain test procedures, 

test reports, discrepancy reports and test results, and facilitate an independent 
review of this package by the customer; 

• Verify compliance by the test team with stated management procedures for 
change control, discrepancy reporting, test reporting, etc. 


An audit during the pretumover period should be conducted at a key point during the testing phase to allow 
time for corrective actions prior to software product demonstration to the customer. In addition, this audit 
should not be confused with the inspection and test surveillance function described in the next section. The 
inspection and test surveillance function is a part of the daily QA involvement with the test activities and 
differs from an audit, which has contractual connotations and provides definite corrective actions and 
direction to the test activities. 


At the conclusion of the final test phase, the software QA organization certifies that the computer programs 
satisfy their specified contractual requirements. This certification is done prior to the turnover of the soft- 
ware to the customer/user organization. 

Inspection and Surveillance of Formal Tests 

Inspection and test surveillance is an on-site activity performed by QA personnel during program 
testing. Specific tasks include : 

• Monitor all tests to ensure that actual tests performed are as specified in 
documented test procedures. This is accomplished via QA signoff on the 
test execution report. 

• Assure that all potential discrepancies are recorded in the approved manner. 

• Compare the configurations of hardware/software components utilized in 
the test against the configuration identified in the test procedure. 

• Certify that analysis of the test output is correct, the test satisfies its intended 
requirements and acceptance criteria, and that the test data package is complete. 

• Assure that master copies of test procedures, test results and test reports are 
maintained and available from a centralized records center. 


Verification of Configurations 


New versions of programs are usually released after formal configuration control procedures have been 
applied. An important QA function is to perform configuration tests on the controlled master libraries of 
each version to assure that no inadvertant or unauthorized code changes have been made. One method is the 
three-step method currently being used to carry out this function: 1) isolate the approved modifications, 

2) delete those modifications from the new master file and replace any removed code (this is done automat- 
ically by a library management system), and 3) compare the old file and the rearranged new file. (An opera- 
ting system utility does this most efficiently). They should compare identically. Any discrepancy should be 



brought to the immediate attention of the configuration control manager for corrective action. A letter 
certifying the authenticity of the new file may accompany the new release notice. Other configuration 
verifications may be incoming and outgoing shipment packages where QA personnel inspect the product 
packages to assure that the contents are as specified. 


Management of the Discrepancy Reporting System 


An important management control mechanism is the accurate recording, tracking, and reporting of all 
real and perceived project discrepancies. The Quality Assurance organization with its independent, project 
wide visibility is often the prime candidate for this job. As discrepancy reports (DRs) are written, a copy 
should be given to the QA office for logging. As is often the case, the original DR gets sent to a central man- 
agement review body who decide whether the DR should be rejected, deferred, or accepted, and if so, who 
should fix it. By making QA a member of this body, they can track each DR and report its status at any 
point in time. Periodic error trend analyses can then be made which report, for example: 

• The error rate for various programs 

• The number of open DRs 


• The relative frequency of DRs in various defect categories 


• The average time to close various problems 


Retention of QA Records 

The previous seven QA functions often create voluminous paper, much of it contractual in nature. To 
properly retain and control this paper a repository with formal procedures needs to be established early in the 
project life cycle. On small projects this may involve only a secretary with a locked file cabinet, while on 
larger projects QA may make use of a centralized data management center. In the quality planning stage 
during proposal or contract definition, the appropriate QA records should be defined and resources budgeted 
for this control. These records may include: 

• QA Plans 

• QA Procedures 

• Design Problem Reports 

• Discrepancy Reports 

• QA Audits and Reviews 

• Test Data Packages 

• Waivers and Deviations 


ORGANIZATION 

At TRW, software quality assurance functions are performed in an organization called Product Assurance 
(see Figure 1 ) which is headed by a vice president level staff director. The dual role of the Product Assurance 
organization is quality assurance and configuration management, as it has been discovered that both functions 
share common characteristics. 

1. They perform staff oriented functions. 

2. The performance of their functions is often times more credible when 
done by an independent organization. 

3. Staff personnel share many aptitudinal characteristics (e.g., close attention 
to detail, preference for wide visibility tasks). 

The project PA staff (Figure 2) is often headed by an Assistant Program Manager (APM) for Product 
Assurance who has responsibility for two project functions: quality assurance and configuration management 
The APM for Product Assurance receives project direction from the project manager, yet he and his staff 
retain their organizational independence from the project by reporting functionally to the Product Assurance 
Organization. 




Figure 1 . Corporate Organizational Structure 



Figure 2. Project Organizational Structure 


BENEFITS 

The benefits of software QA seem to accrue exponentially with the size of the project. This may be 
caused by the fact that the larger the project size is, the further removed the project manager is from the 
development activity. The benefits that TRW has received are: 

1. Provides increased management visibility into the development process through 
reviews and audits. 

2. Reduction of project risk through better requirements traceability and a more 
disciplined and thorough testing. 

3. Enforcement of software standards. 

4. Centralization of software tools development and maintenance. 

5. Centralization of QA records (problem reports, deviations and waivers, reviews 
and audits, test inspection reports, etc.). 

6. A skills center for personnel with multi-project visibility who are better able to 
prepare project plans and procedures. 

7. An independent group assuring that deliverable items meet contractual 
requirements. 














LESSONS LEARNED 


Probably the largest lesson learned is that one key to the successful development of software is the 
employment of a strong QA activity. It is TRW’s experience that the benefits are well worth their price. 
Specific lessons learned with regard to the implementation of a QA program are: 

1. Be sure there is adequate QA participation during the proposal and contract 
definition periods. 

2. It’s easier to hire personnel knowledgable in software and train them in QA, 
than to hire QA personnel and train them in software. 

3. Plan to perform the first audit very early in the development process. This 
leaves adequate time for corrective action. 

4. Announce your intention to audit well before it occurs. The object of an audit 
is not to find problems, but to assure that it was done right. 

5. Prepare for audits by constructing an audit checklist, and distribute the checklist 
to the area to be audited during the announcement of the intent to audit. This 
may eliminate subjective assessments by various auditors, and informs the audited 
party of the exact scope and depth of the audit. 

6. Develop a deep and thorough relationship with each project sub-group by assigning 
QA engineers on a long term basis. Allow QA engineers to colocate with the soft- 
ware developers so they can more fully understand the daily problems of other 
project members. 

With regard to software tools, specific lessons have been learned, and are discussed separately. First it 
is important to state that tools by themselves do not produce reliable software. This disclaimer is important 
because there have been both managers and programmers who are quick to blame a tool for the failure of 
their software to meet a particular goal. Second, the experience with tools at TRW has shown that program- 
mers are curious by nature and are eager to use tools so long as the tools are user oriented, and their output 
is helpful to the programmers. Third, design, development, test, and documentation effort of tools should 
entail the same rigor as deliverable software. While this may seem superfluous to some, one must realize that 
except for very special one-time quickies, the hope for most tools is that they become a permanent part of 
a tools library and that they become available for use by many projects and organizations. As such, the tools 
will be widely distributed and modified to fit specific project peculiarities. One quick way for the tool to die 
is for its maintenance and modification to be made impossible by poor or nonexistent design documentation 
and in-line comments. Fourth, tools should be coded in a higher order language and in such a way as their 
portability from one computer system to another not entail a major rework of the tool. System peculiar 
functions (for example, bit manipulation, file handling, input/output, etc.) should be modularized and well 
documented. Fifth, tools should be designed with as much flexibility as possible in three areas: input 
options, processing options, and report options. Input flexibility is important so that the tool can be used 
on several projects and/or programming languages. Processing options allow the user to guide the tool to 
the specific problem area, or to solve a problem under an outside constraint (e.g., cost of each run, memory 
limitations, etc.). Report options are important because besides eliminating expensive print, it focuses the 
reader’s attention on the information he requested. 

CONCLUSION 

In the early years of its existence, the software QA office spent most of its energy defining those areas 
where it could benefit the software development processes most effectively. The above list of QA functions 
is a derivative of this effort. In 1972, QA began its first dedicated support of a software project: a five 
year, $100 million program. Currently, there is dedicated QA support on almost every large software 
project at TRW. Last year the Systems Engineering and Integration Division’s executive office formally 
published a set of comprehensive software policies and procedures, many of which were derived from QA 
recommendations. These policies and standards are now being followed by all new software projects in that 
division. The Product Assurance organization considers this a significant achievement. 
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INTRODUCTION 

Several approaches to aid in the design, implementation and debugging of large- 
scale software have recently emerged. Examples are restricted modularization (14) , 
structured programming (14), and program verification (9,10). However helpful they 
may be to programmers and low-level managers, the effects of these techniques cannot be 
utilized throughout a software project management hierarchy since they are qualitative 
rather than quantitative: managers should not be expected to understand code and/or 
sophisticated mathematics. 

In this paper we explain how an important phase of software development, testing, 
can be managed effectively by use of the new program testing approach known as program 
mutation (15) . Program mutation provides as a side effect the qualitative type of in- 
formation that managers need to monitor software development and personnel performance. 
The basic idea is: given a program module and its test data, program mutation provides 
a measure, in terms of a percentage, of how "well" the data actually tests the module. 
The higher the percentage, the more adequately the program has been tested. A program 
mutation system produces the percentage and users increase the measure by either aug- 
menting the data in a controlled fashion or by answering "hard" questions about the 
module which are posed by the system. This process iterates until a satisfactory 
testing percentage is obtained. Meanwhile, the program mutation system records all 
the involved information in a data base which can be querried at any time by members 
at all levels of the project hierarchy to obtain reports containing relevant informa- 
tion on the project's testing status. For example, the project manager may wish to 
know only the testing percentages of all program modules while a programmer may wish 
to review in detail some or all of the questions and answers previously recorded for a 
given module. 

In section 2 we detail the theory of program mutation as a program testing tool. 
Section 3 explains what types of information various members of the project hierarchy 
would draw from the mutation system and how that information would be used as a manage- 
ment tool. These concepts are illustrated in terms of a hypothetical compiler con- 
struction project. Finally, in section 4 we present another application of program 
mutation: monitoring software procurment. 

THE PROGRAM MUTATION METHODOLOGY 

Program testing is an inductive science which addresses the following fundamental 
question : 


"If a program is correct on a finite number of test cases, 
is it correct in general?" 

Finite test data which implies general correctness is called adequate test data and 
since adequate test data cannot in general be derived algorithmically (4) program 
testing cannot be deductive. Recently, path analysis (1,2, 5,6) and symbolic execution 
(7,8) have emerged as methods which allow one to gain confidence in one's test data's 
adequacy. Although as with any inductive science, it is possible to make false infer- 
ences with path analysis (3) , the basic idea is undeniable: test data which exercises 
all flowchart control paths of a program at least once must be better than test data 
which doesn't. Symbolic execution is associated to path analysis since, among other 
things, it attempts to derive test data which exercises all paths of a program. 

Unlike previous software reliability methods, in program mutation we make the 
© 1978 American Society for Quality Control 



following assumption: 


Experienced programmers write programs which are either correct 
or are "almost" correct. 

That is, in the mutation terminology, 

If a program is not correct, then it is a "mutant" - it differs 
from a correct program by simple well-understood errors. 

There is empirical evidence which supports this natural premise (11) . 

Boehm has found (12) that errors fall into three categories: clerical, logical, 
and misunderstanding of specifications. In the above assumption we do not explicitly 
mention errors due to programmers misunderstanding specifications; rather, it appears 
we are dealing exclusively with clerical errors. While a system which would solve the 
clerical error problem would be quite useful, program mutation does even more: indeed, 
below we explain how the use of the program mutation methodology can lead to the detec- 
tion of all three error types . 

With the "experienced programmer assumption", the mutation method is: take a pro- 
gram P which is correct on some test data T and subject it to a series of mutant opera- 
tors, thereby producing mutant programs which differ from P in very simple ways. For 
example, if 

I = 1+1 

is a statement in P, then 

I = 1-1 
I = 1+2 

I = 1+0 (i.e., a no-op) 

are all simple changes which lead to three mutants of P. The mutant programs are then 
executed on T. If all mutants give incorrect results then it is very likely that P is 
correct (i.e., we can infer with high confidence that T is adequate). On the other 
hand, if some mutants are correct on T then we can infer that either: 

(1) The mutants are equivalent to P, 

(2) The test data T is inadequate, or 

(3) The program P is incorrect. 

If it cannot be determined that P is incorrect from this information, then T must be 
augmented and the mutation method re-applied in an attempt to make the non- equivalent 
mutants which are correct on T subsequently fail. This augmentation process forces 
the programmer to examine P in detail with respect to the mutants. 

At first glance it would appear that if T is determined adequate by mutation anal- 
ysis , then P might still contain some complex errors which are not explicitly mutants 
of P. To this end there is a coupling effect which states: 

Test data on which all simple mutants fail is so sensitive that it is 
highly likely that all complex mutants must also fail. 

That is, if a program passes tests for all possible simple errors then it has been 
implicitly tested for all possible complex errors. It is in this effect that the 
power of program mutation to detect the so-called logical errors of Boehm (12) is 
revealed. Experiments which substantiate the coupling effect are reported in (13). 

Using program mutation as a tool for obtaining reliable software is a highly 
interactive process whose success depends in part on human judgement. Due to the 
following issues , the programmer must re-examine in critical detail both his program 
and its specifications and why he made the decisions that led to the construction of 
his program. The crucial issues which must be addressed by the users include: 

(1) Which mutant operators should be applied to the program? 

(2) Are the program and its mutants correct on the given test data? 

(3) Is a given mutant equivalent to the program? 

It is here that specifications errors are discovered. Note that it is possible for a 



mutation system to provide the users with information which greatly facilitates resol- 
ving these issues: indeed, a mutation system can even resolve them automatically in 
some cases. 

In using a program mutation system, a programmer specifies to the system his pro- 
gram, test data, and the mutant operators he wishes to be applied. The system then 
generates and executes the mutants on the test data and produces a report indicating 
which mutants are correct on the given test data. The determination of mutant correct- 
ness is done in one of two ways: (1) by direct comparison of the mutant output with 
the program's output, or (2) by a user-supplied algorithm which examines the output of 
the mutant. In both cases the system asks the user whether or not the program is 
acceptable on the test data. However, determination of mutant failure is done by the 
system. 

Upon examining the report, the user may re-run the system and augment his test 
data in an attempt to make the remaining mutants fail. He may also specify that addi- 
tional mutant operators be applied to the program. The system produces another report 
of the same nature as the first for the user to examine. This cycle continues until 
the user is satisfied that his current test data adequately tests his program. 

MANAGEMENT ASPECTS OF PROGRAM MUTATION 

Successful large-scale programming projects rely on a hierarchical flow of infor- 
mation and decisions. A fragment of such a project structuring is represented in 
figure 1. In addition, there is a recognizable time-ordering of events for gathering 
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Figure 1. Hierarchical Management Organization 


information and making decisions which correlates with the hierarchical management 
structure. Events such as "decide input file structure", "gather documentation from 
the submodules of module Ml", "begin testing module M5" provide transformations of the 
programming task, replacing the as yet incomplete project with the next stage as deter- 
mined by the most current information. The management hierarchy generally parallels 
the modular decomposition of the programming task. This can be seen directly in 
figure 2 where we illustrate a decomposition of a multiple pass compiler. 

During the test phase of the project the mutation system records a wealth of 
information in its data base and this data is used to produce reports which directly 
influence decision-making throughout the project hierarchy. The type of information 
drawn from the mutation system and its uses vary depending on the project hierarchy 
level of the querrier. In this section we sketch some possibilities for the three 
levels illustrated in figures 1 and 2. Additional possibilities can readily be imag- 
ined. The general idea is: the higher the querrier is in the project structure, the 
less programming oriented is the gathered information. 

Project Manager's Report 

The project manager periodically meets with the chief programmers to evaluate the 
project's testing status. Also, the assignment of personnel and the evaluation of 
personnel performance are done at this level. The project manager's report would con- 
tain information such as : 


(1) The name of each module. 
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Multiple Pass Compiler 


(2) The chief programmer responsible for each module. 

(3) Plots of the mutant elimination percentage vs. time for each submodule. 

(4) For each submodule, (a) the number of mutants, <b) the number and the 
percentage of eliminated mutants, (c) the number and percentage of mutants 
deemed equivalent, and (d) the number and percentage of non-eliminated 
mutants . 

(5) For each module, the number and type of assigned personnel. 

(6) For each submodule, the number and type of assigned personnel. 

This information can be used by the project manager to help do the following: 

(1) Monitor adherence to the project's testing pert-chart. 

(2) Decide whether an acceptable level of testing has been obtained for a 
given module or submodule. 

(3) Re-assignment of personnel to work on modules where the mutant elimination 
percentage is low. 

(4) Rewarding personnel who achieve high mutant elimination percentages. 

(5) Pinpointing responsibility for modules which fail after having been judged 
acceptable . 

Chief Programmer 1 2 3 4 5 6 7 s Report 

A chief programmer should be familiar with the program code of all the submodules 
but he doesn't necessarily do any of the programming himself. He meets daily with his 
subordinate personnel. The type of information contained in a chief programmer's 
report would include: 

(1) The names and program code for each submodule of his module. 

(2) The personnel assigned to each submodule. 

(3) Plots of the mutation elimination percentage vs. time for each submodule. 

(4) The mutant operators being applied to each submodule. 

(5) For each submodule, (a) the number of mutants, (b) the number and the 
percentage of eliminated mutants, (c) the number and percentage of mutants 
deemed equivalent, and (d) the number and percentage of non-eliminated 
mutants . 

(6) Listings, in coded forms, of mutants determined equivalent. 

(7) Personnel responsible for classifying mutants as equivalent. 

This information can be used by the chief programmer to do the following: 

(1) Suggest to the programmers additional mutant operators for a given 
submodule. 

(2) Ask a programmer to justify his judgement of mutants as equivalent. The 
chief programmer may want to know, for instance, why it does not matter 
if a certain variable can be mutated without changing the effect of the 
submodule. That is, why is his submodule so insensitive to that mutation? 



(3) Determine that a given submodule has been acceptably tested and 
prepare evidence on this decision for presentation to the project 
manager. 

Programmer's and Tester's Report 

These personnel are concerned mainly with the details of program code and data 
and thus their reports will be the most lengthy. The type of information would 
include: 

(1) A listing of the submodule code. 

(2) The current test data for the submodule. 

(3) The mutant operators currently being applied to the submodule. 

(4) For the submodule, (a) the number of mutants, (b) the number and the 
percentage of eliminated mutants, (c) the number and percentage of mutants 
deemed equivalent, and (d) the number and percentage of non-eliminated 
mutants . 

(5) Profiles of the information in (4) with respect to the mutant operators 
currently being applied. 

(6) Listings, in coded form, of the non-eliminated mutants. 

(7) Listings, in coded form, of the mutants determined equivalent. 

This information could be used by programmers and testers to do the following: 

(1) Augment the current test data so as to eliminate mutants on the next 
mutation run. 

(2) Augment the set of applied mutant operators for the next mutation run. 

(3) Classify non-eliminated mutants as equivalent. 

(4) Determine that the submodule has been adequately tested and prepare 
evidence of this for presentation to the chief programmer. 

SOFTWARE PROCURMENT ASPECTS OF PROGRAM MUTATION 

Government agencies and profit making industries are currently finding that pur- 
chasing software from specialized software vendors is more economical than in-house 
development. The contracts generally consist of the specifications for the software 
and a date on which the software and test data on which the software meets the speci- 
fications are to be delivered. Occasionally, some test data is given with the speci- 
fications. 

Two problems for the contractor are apparent in the above scheme: (1) at any 
time during the contract period the puchaser has no indication as to how "close" the 
software is to being ready, and (2) upon delivery, although the software works correct- 
ly on the supplied test data, there is no way to measure the quality of the purchased 
software. We see program mutation as a partial solution to the first problem and as a 
definite solution to the second. 

Since program testing is the final stage of software development, a contractor 
can specify that the vendor indicate at what point testing commences. Assuming that 
the vendor is using a mutation system, the contractor can monitor the final stage of 
development by having the vendor periodically report mutant elimination percentages. 

To evaluate the delivered software, one can specify in contracts that the test 
data of modules must eliminate a certain percentage of the mutants with respect to 
"standard" mutant operators. Here there are many options. Software not passing this 
quality test may be rejected or there could be a substantial financial penalty to the 
vendor. In this case it is not essential that the vendor use a mutation system, only 
that the contractor have one available to evaluate the final product. Also, note that 
the contractor is not concerned with equivalent mutants; rather, a simple test (which 
can be entirely computerized) dependent solely on the mutant operators is used. Cur- 
rently, we have little information on which mutant operators should be employed in this 
test; however, experiments to answer this question are underway. We have observed 
empirically (13,15) that the percentage of equivalent mutants tends to be about two. 

SUMMARY 

Program mutation is an important new tool in the field of program testing which 
has applications in other fields. Above it has been explained how, unlike other cur- 
rent programming methodologies, a program mutation system can provide quantitative 
information which can be used throughout the management hierarchy of a large program- 
ming project. Furthermore, program mutation has an important application in that it 
can be incorporated into contracts for software procurment. It provides purchasers of 
software with a means of measuring the quality of the delivered product. 
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QUALITY PERFORMANCE CAN BE RATED 


Thurman J. Slack, Manager of Quality Assurance 
Pako Corporation 
Minneapolis, Minnesota 55^0 


Programs are being developed to exploit many published philosophies. To name a 

few : 

1. Zero Defects ( Z D ) ^programs are built around the concept that human error 
need not be inevitable. 

2. Behavioral modification is often based on supervisory attention on what is 
right with quality - related performance. 2 

3. Time oriented action plans by an identified individual toward a specific 
goal are the necessary ingredients of programs that follow the "Management 
by Object i ve"3out 1 i ne. 

A. Subspecies of operator error (inadvertent, technique, and wi 1 1 ful ) ^breaks 
away from the idea that motivation is a cure-all for operator controlled 
defects and suggests a good diagnosis of operator errors is necessary. 

The one common thread in all these is people. Everyone attempts to influence 
people to be sensitive to quality matters under the assumption that people will react 
positively to quality goals and that quality levels can be improved if they can be 
measured and tracked. 

However, how often the cry "the quality was bad on the last lot but: (1) The in- 
formation was too late and, (2) Jt didn't have the detail necessary to correct the 

problem". 

These statements underline the basic problem of programs that collect information 
but do not adequately communicate it so that timely corrective action can be taken. 
Many times the course of action is perfectly clear to the author of the report but 

does not follow the basic truth that was expressed by the late Harold F. Dodge at his 

address to the A.S.Q.C. 27th Annual Technical Conferences. "If you want a method 
used, keep it simple." 

This paper describes a successful program that provides timely information to the 
production people and makes it simple enough for everyone to understand. It is sup- 
ported by the Quality Creed issued by the president of the company. The opening 
statement of the Creed is "More than any other single factor, the quality of Pako 
products, parts, and accessories and the degree of customer satisfaction with these 
products will determine the ultimate growth and future of our corporation". 


OBJECTIVE 

The objective of the quality rating system at Pako was to establish a daily/ 
weekly working tool for production and a trend analysis report. For management the 
longer range objective was to establish the quality level base so that a Breakthrough 
goal could be set and programs for accomplishing it could be developed. 

More specifically, the quality rating had to do all of the following: 

1. Be consistent regardless of who reported the information 

2. Be insensitive to machine complexity 

3- Be understandable to all levels in the company 

4. Be able to identify quality trends in a daily, weekly, monthly, and 
quarterly basis 

5. Be able to analyze quality trend In the following matrix: 

5.1 By defect category 

5-2 By machine 
5<3 By foreman 

^ 1978 American Society for Quality Control 



COMPANY PROFILE 


Pako produces specialized equipment and controls used in the processing, print- 
ing and packaging of sensitized materials for the photographic, graphic arts, and x- 
ray fields. The equipment specifications are highly dependent on changes in the sen- 
sitized materials industry, resulting in a need for multiple models and frequent de- 
sign changes in equipment produced by Pako. 

The company has a long standing good quality image. This image was unfortunate- 
ly too often protected in the field by fixes made by the field service organization 
during installation before the customer was aware of any problem. These fixes sig- 
nalled the need for quality improvement. 

Another factor that demanded a more organized approach to quality was the rapid 
expansion of highly complex products being developed by Design Engineering. The mar- 
ket demanded more speed and features, thus, electronics were being substituted for 
the electro-mechanical controls. 

Executive management set an objective: Establish quality assurance at Pako. 

The objective was complicated by the fact that Pako is a rather complex company. 
It manufactures an extremely wide variety of equipment and utilizes many manufactur- 
ing technologies under one roof. While not a complete list the following illustrates 
my point. 

1. The functional requirements of equipment manufactured include: 

1.1 The positioning and cutting of film and prints at the rate of 7 per 
second 

1.2 The transporting of film and photographic paper through chemistry with- 
out damage 

1.3 The sensing, color and exposure correcting, and exposing of photographic 
prints at the rate of 3000 an hour. 

1.4 The automatic sorting, pricing, invoicing, and packaging of film, prints 
and advertising material 

1.5 The ability to function over many years in an environment of highly 
corrosive photographic chemicals 

2. The manufacturing technology employed includes: 

2.1 The use of tape controlled and standard types of sheet metal and weld- 
ing equipment on cold roll steel, aluminum, stainless steel and 

t i tan i urn. 

2.2 The use of tape controlled and standard types of machines for machining 
and grinding cold roll steel, aluminum, stainless steel, castings, 
plastic, phenol ics, and brass to tolerances as close as + .0002. 

2.3 The use of electrostatic painting and other standard finishing equip- 
ment for epoxy painting, plating, anodizing, passivating, black oxide, 
and electro-polishing of parts. 

2.4 The use of programmable and standard electronic test equipment for in- 
specting and troubleshooting TTL, CMOS; and microprocessor type P.C. 
boards as well as electro/mechanical control panels. 

Quality improvement was needed, and it had to be done at a minimal initial in- 
crease in cost. The long range goals were to reduce cost (as a percent of sales) 
and improve quality. 


QUALITY RATING BASIS 

The basis of the Pako's Quality Rating system is to relate reportable errors on 
a weighted base. A minor error is defined as something that might irritate a custo- 
mer but he could still get his machine in operation without any delay. A major error 
is one which could cause delay in getting a machine operational but which could be 
repaired with available tools and parts. A critical error is one which causes a ma- 
jor machine failure and could require major part replacement or even the replacement 
of the entire machine. 



The quality rating (Q.R.) is the combination of the number of defects found on 
the machine as received from the assembly line, the weighted multipliers assigned to 
the type of error, times the correction factor assigned to each machine and dependent 
on its complexity. The results are expressed as the estimated percentage defective. 
This is subtracted from 100 so that the positive effect of reporting good units is 
reflected in the quality rating. (Fig. 1) 

This numeric rating is further categorized into narrative levels of quality. 
(Note: These are listed on the left side on graph, Fig. 1) 

The unique part of this rating system is the way the correction factor for ma- 
chine complexity was developed. Ideally, the total bill of material, assembly tasks, 
type of equipment, and numbers of different technologies should be quantified. How- 
ever, this is very costly and time consuming; therefore, the following technique was 
used . 

1. All types of machines were reviewed as to the number of potential inspection 
error categories that were represented. (Table l) 

2. The machines were reviewed as though they were of modular construction, ?.e. 
electrical panel was treated as though it were a complete unit within itself. 
Thus, for example, a wiring error could occur in the panel as well as on 
component parts of the machine thus multiple wiring errors could be reported 
on one mach i ne . 

3. Selected machine types were rated in order of descending complexity by 12 
people who had a working knowledge of all company products. (Table ll) 

4. This list was summarized and the total inspection points (categories listed 
below) were adjusted so that the total potential quality impact (sum of cate- 
gories/times/impact multiplier) was consistent with the machine complexity. 


Table I. INSPECTION CATEGORIES 



Mi nor 


Major 


Critical 

1 . 
2. 

Appearance, dents, scratches 
Neatness of harness/wiring 

6. 

Improper electrical 
connections 

9. 

Hi-pot failure 
assembly error 

3. 

4. 

Missing items 

Fastenings loose/missing 

7- 

Adj ustments 
i ncompl ete 

10. 

Mi swi red electr ica 
conrtect i on 

5. 

Plumbing error 

8. 

Leaks 

1 1 . 

Meehan i ca 1 
ma 1 f unct i on 


Table 1 1 . MACHINE 

COMPLEXITY 


Mach i ne 

Total Impact 

Control Factor 

Photopacker 

244 

.41 

Mach 1 1 Printer 

212 

.47 

Cine Film P rocessor 

207 

.48 

Graphic Arts Processor 

148 

.68 

B/C 24 Printer 

118 

.85 

Medical X-ray Processor 

76 

1.32 

Micro-Film Processor 

67 

1 .49 

Print Dryer 

67 

1 .49 

Washer 

37 

2.70 


The total impact was divided into an arbitrary base of 100. The resulting num- 
ber was called the control factor. The purpose of this control factor was to par- 
tially reduce the effect of each error on the quality rating of complex machines and, 
likewise, increase it on simple machines. 

The major significance in recognizing machine complexity comes in selling the 
number to the production foreman. If he knows that his complex machines with hun- 
dreds of wires will not be impacted by a single error as much as a simple machine, 
he will agree to the quality rating number. Once this occurs, management can use 
the numbers for performance reviews. Quality then becomes as important to the fore- 
man as his labor performance. 




Figure 1. Equi pment/Assembly Quality Report 


One question comes up often. Are all errors charged to the assembly foreman? 

No, only errors that were considered his work are charged to him. For'example, a 
leaking tank is not the fault of assembly but a leaking hose fitting to the tank is 
h i s respons i bi 1 i ty . 

However, the component failures are not ignored. Part numbers are included on 
the inspection sheets and are logged by the Quality Engineering Coordinator. Repeti- 
tive part numbers are flagged for possible corrective action. This is an aid in de- 
fining receiving inspection and fabrication faults. 

The equipment/assembly quality report {Fig. 1) is distributed weekly to each 
foreman. The chart includes both a weekly and a three week moving average of his 
quality rating as well as a listing of the errors that made up the weekly rating. 

Monthly a similar report by foreman and by the total assembly department is dis- 
tributed to all management. 


RESULTS 

Based on two full years, the program has tracked the following results. 
(Table 111) 


Table III. 1977 vs. 1 976 Machine Quality Analysis 


Quant i ty 

Qua 1 i ty Rating 

Errors/Mach. 

Spoi lage 
{% Std . Cost) 

Field Prob. 

Per Mach. 

'll 


'll 

'76 

'll 

'76 

'll 

'76 

'll 

’76 

5450 

4126 

96.42 

96.00 

.58 

66 

1 .46 

2.23 

.09 

• 19 

+32. U 

+ .4% 

-12. U 

-34.53; 

-52.5% 


During this period, the quality rating took three dips. The first one was ob- 
served and communicated to the production organization. The trend continued until 
middle management implemented a corrective action program. As the problems diminished 
the quality rating raised and confirmed that the corrective action was successful. 

The second dip appeared during a heavy six week production load. The reaction by pro- 
duction was immediate with a resulting good quality level being maintained during the 
remaining four weeks of heavy production. 

The third dip was observed approximately 18 months after the Q.R. program was in- 
itiated. This dip was interpreted by management as a status quo condition that could 
become serious if ignored. Quality management reacted by embarking on a long range 
quality awareness program. 

Introduction of the quality awareness program was aided by using the results of 
the quality improvement breakthrough that was achieved on a particularly troublesome 
machine. The recorded errors per machine were very high on this model even though it 
was in continuous production. The results were as follows in Table IV. 


Table IV. Improvement on a particularly troublesome machine 



1 

2 

3 

ERROR 
4 5 

CATEGORY 

6 

(See 

7 

Table 1) 

8 9 

10_ 

1 1 

Tota 1 

1st 6 mo . 
(598 units) 

0 

0 

51 

110 

3 

23 

36 

128 3 

40 

8 

402 

2nd 6 mo. 
(586 uni ts) 

1 

1 

27 

33 

6 


n 

61 4 

a 

■ 

167 

3rd 6 mo. 
(442 units) 

0 

0 

5 

8 

6 

4 

0 

70 1 

12 

B 

107 

































During that period, the quality rating went from under 95% to over 33 %. 

Based on the success of improving the Q.R. of this machine, the time had arrived 
to embark on J. M. Juran's Breakthrough philosophy. The Quality Rating had proven 
that the production organization was under control. The time for a Breakthrough into 
new levels was also appropriate since the measurement tools had been proven and the 
foreman and the quality control personnel had faith in the Q.R. numbers. 


ADDITIONAL BENEFITS 

The technology of the rating system and the reports have been made applicable to 
other areas of the company. Examples included the following areas: 

1. Stockroom, shipping, warehouse and other materials functions 

2. Fabrication departments (by foreman and work center) 

It has provided a means for recognizing good work and provides tools for future 
i mprovements . 

A secondary impact has been realized for agencies such as OSHA and UL. The re- 
cords and corrective action programs generally indicate to an outside inspector that 
the company is responsive. This alone has built confidence and reduced outside in- 
spector involvement with the company. 

The evidence of improvements in quality trends also aided in more effective man- 
power utilization. No longer does a single reported field failure create an unjusti- 
fied "Quality Improvement Program" at the expense of preventative efforts. 


SUMMARY 

The Pako Quality Creed also contains the statement "The true existence of quality 
control is in the attitude and commitment demonstrated by all employees". 

The idea of making quality everybody's business is beginning to unfold. With it 
is the theme "You Have a Hand in Quality". 

Tied to the theme is the quality rating system which has the ability to recog- 
nize those who achieve and help those who don't. 
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ACCEPTANCE SAMPLING DECISION RULES FOR ATTRIBUTES 


David C. Whitford 
Statistical Methods Division 
U. S. Bureau of the Census 
Washington, D. C. 20233 


INTRODUCTION 

In census and survey data processing many quality control rules have been used to 
assure continued quality after initial qualification in the work of coders, typists or 
keyers. An example of such a rule is, "A worker remains qualified if s successive 
work lots are accepted within a maximum of d inspected." The purpose of this paper is 
to summarize the probability distributions and tables associated with some decision 
rules of this type and to examine a new rule which has recently been used in Census 
Bureau quality control decision processes. This paper reviews and continues publi- 
cations on the same topic by George Minton and others. A selected bibliography is 
included. 


ASSUMPTIONS 

If our basic production unit is called a lot (of N items), a decision regarding 
acceptance or rejection of the lot is made on the basis of a sample of n items taken 
from the lot. We assume n to be small with respect to N so that the binomial distri- 
bution can be used to find the probability that a single lot is acceptable. Grubbs 
(1949) states that the lot size should be 10 to 15 times the sample size before the 
binomial distribution becomes an excellent approximation to the hypergeometric distri- 
bution (the exact distribution in this case) . 

A second assumption is, that the lots themselves are of homogeneous quality with P 
as the true fraction defective. 

Finally, we assume that inspection is error-free. In practice this assumption is 
approached only in most independent error detection plans. Quality control models 
which incorporate Type I inspection error (an inspector classifies a nondefect as a 
defect) and Type II error (an inspector classifies a defect as nondefective) are 
discussed in Lavin (1946), U. S. Bureau of the Census (1965), and Minton (1969). 

Using these assumptions, the probability of accepting a work lot is 


L p = 

where c is the acceptance number and Q=1 - P. 

DECISION RULES 



( 1 ) 


Rule 1: 

A worker remains qualified (survives the decision) if f or fewer work lots 
fail sample inspection within d inspected. 

If the probability of a lot failing inspection is (1-Lp) , then the probability of 
having f or fewer failures in d lots (call this is computed from the binomial 

distribution: 

where i is the number of rejected lots failing sample inspection 
(i=0,l, . . . ,f). 


<^V 


( 2 ) 


©1978 American Society for Quality Control 


Given the arguments L , d, and f , the probability of a worker's survival is easily 
found using a cumulative binomial table. Minton (1970) provides such a table. 

Rule 2: 


A worker survives if s successive work lots are accepted within a maximum 
of d inspected. 


The probability of a worker's survival after d work lots under this rule is not 
as straightforward as in Rule 1, but Burnside (1959) provides a recursion formula 
which Minton (1970) uses to construct a table of survival probabilities for this rule. 
The formula for Qj s the probability of s successive acceptable work lots within a 
maximum of d inspected is: 


r-1 


(i+l)s 


Qd,s = L P + .^MiCLp) a- L p) - (Ni) (Lp) (1-bp) (3) 

d 

where r is the largest integer in - 


Mi = (-D 


i+1 d-is 


N. = (-1) 


i+1 d- (i+1) s 


Investigation of the tabular survival probabilities for Rules 1 and 2 indicates 
that comparable probabilities of acceptance can be attained if d is permitted to differ 
between rules. As opposed to the following Rules 3 and 4, these first two decision 
criteria require quality assessments only after multiples of d work lots are completed. 
This has two disadvantages: (i) after each d lots a worker has no credit (or debit) 
for previous work performed and (ii) a decision on quality cannot be made until d lots 
are completed. These effects might lead one to consider the following rules which 
potentially allow (except at the outset of production) a work quality decision after 
each work lot completed. 


Rule 3: 


Given C points initially, a point is added when a worker's lot is accepted or 
deducted when a lot is rejected. If the point total remains greater than zero 
while working D lots, the worker remains qualified. 


The probability of surviving D decisions given C initial points is originally 
suggested by Feller (1957) and is derived by Cook (1961) . Minton (1970) used this 
work to provide a table of survival probabilities for this rule. The probability 
of survival (called U ) given D,C, Lpis: 


U 


D,C 


D 

£ 

i=F 




D-i 

o-y 2 


(4) 


where F=D-2 

[r I 


*[[(D+C-l)/2 ]] and 

indicates that the greatest integer in 


D+C-l 

2 


is to be used. 


Rule 3 has the advantage that further study has been made concerning it. Minton 
and Krivitsky (1971) give a table for the expected number of points given continuance 
after D decisions and a table for the expected number of decisions given worker re- 
moval before D decisions. However, a disadvantage of Rule 3 is that a worker might 
accumulate an abundance of points thus insuring his/her continuance even if subsequent 
work deteriorated. This drawback plus the previously mentioned disadvantages of 
Rules 1 and 2 has lead to consideration of Rule 4 (the "new rule" mentioned earlier). 


Rule 4: 


A worker remains qualified if f or fewer work lots fail sample inspection 
within the last d inspected. 



This criteria (i) provides for a work quality decision after each lot is com- 
pleted, (ii) always considers previous work in the decision, and (iii) always decides 
only on the basis of the most recent work produced. Considering these attributes 
an attempt was made to find a closed form for the probability of survival using the 
arguments Lp,d,f and D (the number of the last lot worked). But due to the conditional 
nature of the rule, derivation did not seem as feasible as computer enumeration of 
possibilities. This method has produced Table I which gives ^ the probability of 
survival after D lots based on whether f or fewer lots fail inspection within the last 
d lots inspected. Probabilities are given for Lp=.05(.05)1.00 and survival probability 
curves are presented in Figures 1-4. 


SUMMARY 

In examination of Table I and Minton’s (1970) tables for Rules 1, 2, and 3 it 
can be seen that comparable survival probabilities can be attained using any of the 
four rules if d is permitted to differ between rules. In choosing a decision rule, 
though, the value of d and the survival probability curve might be of primary impor- 
tance. Then the various advantages/disadvantages mentioned here would be of little 
consequence; the best rule would be found just by examining the tables. However, 
these four rules, their probability distributions, Table I, and some of the advantages/ 
disadvantages of their use might help provide decision rule users with a larger infor- 
mational basis by which to choose a rule. 

MATERIALS 

Copies of the tables, published by Minton (1970), for Rules 1, 2, and 3 are 
available from the author upon request. The computer program used for Table I is 
also available. 
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TABLE I. S' Probability of Surviving D Decisions if at Each Decision Only f 
or Fewer Work Lots Can Have Failed Within the Last d Inspected. 
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CONSTRUCTION AND USE OF QUALITY CONTROL TABLES 


Michael L. Mersch 
Statistical Methods Division 

and 

Thomas C. Dyke 

Statistical Research Division 
U. S. Bureau of the Census 
Suitland, Maryland 20233 


INTRODUCTION 

The Census Bureau has many computer programs to assist in the development of 
sampling plans. Each computer program is for a specific type of sampling plan, i.e., 
single, double, sequential. Dodge, etc. These programs are usually used in conjunc- 
tion with an interactive time-sharing system. Therefore, several people may use the 
same computer program at the same or different points in time, thereby duplicating 
effort. With this in mind, it is felt that a first step toward saving time and money 
plus having more accurate and timely information would be to create and distribute to 
each quality control area within the Census Bureau a set of single acceptance sampling 
plans. The plans are produced based on the Binomial Probability distribution and are 
accompanied by several other tables that would give ranges of sampling plans for 
specific desired AOQL, AQL, and LTPD values. 

QUALITY CONTROL TABLES 

Four separate tables have been constructed to facilitate the design of the quality 
control plans. The four tables are: 

1. Operating Characteristic (PC) Curve Table 

For a given sampling plan (i.e., sample size and acceptance number combination), the 
probability of accepting a lot and the average outgoing quality, as a function of the 
estimated incoming lot quality. 

2 . Average Outgoing Quality Limit (AOQL) Table 

For a given AOQL, the sampling plans that would yield that AOQL. The AOQL is defined 
as the largest calculated AOQ value for the given sampling plan. 

3. Acceptable Quality Level (AQL) Table 

For a given AQL, the sampling plans that would yield a 5 percent a error. 

NOTE: The AQL is used as a basis for some inspection systems and is commonly 
associated with a 5 percent a error or producer's risk. That is, the 
AQL is the incoming error level associated with a 5 percent probability 
of rejection for the given sample size (the compliment of a 95 percent 
probability of accepting the lot) and acceptance number. 

4 . Lot Tolerance Percent Defective (LTPD) Table 


For a given LTPD, the sampling plans (i.e. , sample size and acceptance number com- 
binations) that would yield a 10 percent 3 error. 

NOTE; The LTPD is used as a basis for some inspection systems and is commonly 
associated with a 10 percent 3 error or consumer's risk. That is, the 
LTPD is the incoming error level associated with a 10 percent probability 
of acceptance for the given sample size (the compliment of a 90 percent 
probability of rejecting the lot) and acceptance number. 
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All tables assume that all defects m the selected sample are detected (zero inspec- 
tion error) , and all rejected work units are 100 percent rectified with all defects 
within the work units being detected and corrected. [2] 


SPECIFICATIONS FOR DERIVING QUALITY CONTROL TABLES 


Operating Characteristic (PC) Curve Table 

The Operating Characteristic (OC) Curve Table provides the probability of ac- 
cepting a given lot and the average outgoing quality as a function of the lot quality 
(incoming error rate) for given sample size and acceptance number combinations. 


The probability of acceptance was computed using the Binomial Probability formula 


P c (P) 



X 

p . 


(1-p) 


n-x 


whe re , 

P^ (p) = probability of acceptance 

n = sample size 

c = acceptance number 

p = lot quality or incoming error rate 

Then, given the probability of acceptance, the average outgoing quality (AOQ) 
was calculated using the formula: 


AOQ = p . P c (p) 

NOTE: The Binomial Probability distribution was used as an approximation to the 

hype rgeome trie distribution by making the following assumptions: 

1) the work unit, N, is large, 2) the work unit size, N, is substantially 
larger than the sample size, n, and 3) the binomial distribution represents 
the universe even though one particular individual's or machine's quality 
will vary from hour to hour, day to day, etc. [l], [2], [3], [4] 

Average Outgoing Quality Limit (AOQL) , Acceptable Quality Level (AQL) , and Lot Toler- 
ance Percent Defective (LTPD) Tables 


The Average Outgoing Quality Limit (AOQL) , Acceptable Quality Level (AQL) , and 
Lot Tolerance Percent Defective (LTPD) Tables were all produced using the Binomial 
Probability formula. 

The computer program to compile the AOQL, AQL, or LTPD Tables used the same data, 
but each was sorted separately. Each table used as its primary sort key the AOQL, 

AQL, or LTPD value, respectively. The secondary sort key in all tables was the accep- 
tance number. 

Value Ranges 

The value ranges for the Quality Control Tables are as follows : 


Variable Range Increment 

sample size (n) 10 to 200 1 

205 to 350 5 

360 to 500 10 

0 to [0. 15 (n)+l] 1 


acceptance number (c) 



Table 1 Operating Characteristics (OC) Curve Table 
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Table 3 


Saetple 31 m Renges for given Accept Mice ftusbere end Acceptehle Quality Level 
























Variable 


Increment 


Range 


error rate (p) 

0.005 

0.300 

to 

to 

0.250 

0.500 

0.005 

0.100 

AOQL 

0.005 

to 

0.100 

0.005 

AQL 

0.005 

to 

0.100 

0.005 

LTPD 

0.005 

to 

0.150 

0.005 


For AOQL Tables, the minimum sample size for a particular acceptance number must 
give an AOQL which comes as close as possible (to the nearest 0.0001) to the desig- 
nated AOQL without exceeding it. 

For AQL Tables, the probability of acceptance at the AQL is as close as possible 
but not less than 0.95 (to the nearest 0.0001) for the maximum sample size in the 
range for a particular acceptance number. 

For LTPD Tables, the probability of acceptance at the LTPD is as close as possible 
but not more than 0.10 (to the nearest 0.0001) for the minimum sample size in the 
range for a particular acceptance number. 

Output Formats 

Examples of the output formats for the Quality Control Tables are shown in: 

Table 1 for the Operating Characteristics (OC) Curve Table; Table 2 for the Average 
Outgoing Quality Limit (AOQL) Table; Table 3 for the Acceptable Quality Level (AQL) 
Table; and Table 4 for the Lot Tolerance Percent Defective (LTPD) Table. 

Use of Quality Control Tables 

Each Quality Control Table may be used individually or in conjunction with the 
other tables. For example, if an operation has variable lot sizes and a systematic, 
random sample is being selected, this will thereby yield variable sample sizes. If the 
operation were established on the basis of a 7.00 percent AOQL, then Table 2 would 
indicate a series of sampling plans that would result in the desired AOQL. Further 
information concerning each individual sampling plan; i.e. , probability of acceptance 
at given incoming error rate levels, could be obtained by referring to the appropriate 
OC Curve Table (s). 

The same procedure could be followed if the quality level initially established 
were on an AQL or LTPD basis instead of the above-mentioned AOQL basis. 

COMPUTER PROGRAMS 

All calculations were performed on a Univac 1110 computer. The output data were 
recorded on magnetic tape and the tables were produced on a Xerox 1200 computer/ 
printing system. 


REFERENCES 


1. Brownlee, K.A. , Statistical Theory and Methodology - In Science and Engineering . 

Second edition. New York: John Wiley & Sons, Inc. 1967. 

2. Duncan, Acheson J. , Quality Control and Industrial Statistics . Third edition. 

Homewood, Illinois: Richard D. Irwin, Inc. 1965. 

3. Mosteller, Frederick and Rourke, Robert E. K. , Sturdy Statistics - Non -parame tries 

and Order Statistics . Reading, Massachusetts : Addison-Wesley Publishing 
Company, 1973. 

4. Ostle, Bernard, Statistics in Research. Ames, Iowa: The Iowa State University 

Press, 1963. 


LCS 222:70:991 


EFFICIENT DECISIONS THROUGH RISK EVALUATION 


Alfred P. Aepli, Division Manager Quality Assurance 
Engineering and Maintenance Dept. , 

Swiss Air Transport Co. Ltd., 
Zurich-Airport/Switzerland 


REVIEW OF QUALITY ENVIRONMENT 


Proper management requires a balanced set of clearly defined ob- 
jectives among which quality should be a major one. In Air Transport 
quality objectives may be summarized in safety, regularity and passen- 
ger comfort, all within an economical framework. 

As a next step management decides what has to be done to reach 
these objectives. Although the maxim "quality cannot be inspected in- 
to a product, it has to be built in" is widely accepted by management 
today, the ideal condition, with no inspection required, simply does 
not exist in practice. Some vital quality characteristics, in particu- 
lar those directly related to safety, will always have to be inspected. 

However, to decide which maintenance activities may cause a cri- 
tical situation if improperly performed is sometimes a rather difficult 
task. The incorrect installation of a gasket in the powerplant oil- 
system may not show up on the teststand but result in an engine failure 
later in operation, eventually affecting safety, regularity and passen- 
ger comfort. If the airplane stays on ground at a place where no spare 
powerplant nor adequate lifting facilities are available, then this 
little gasket has caused a real problem. 

Modern Jet Aircraft have developed into very complex systems. 

There we have a structure containing electrical, hydraulic, water and 
airsystems, more and more electronics of advanced technology and high 
performance powerplant attached to it. On the other hand designers de- 
veloped failsafe structures, systems more and more redundant and nume- 
rous built in testcircuits . 

All these technical aspects have to be considered for the planning 
of the inspection function. 

Beside the technological conditions, human factors may be of even 
greater influence on quality. It should be taken into consideration 
that 


- no additional inspection orders are given by 
supervisors merely for their personal protec- 
tion 

- orders for corrective actions are not imple- 
mented the easy way by performing just an 
additional inspection, like an alibi that 
something has been done instead of solving 
the real problem 

- "personel protection" and "alibi"-inspections 
do not become a general law ("it has always 
been done this way") , going on for years with 
nobody remembering the real reason for it* 
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The quality climate may further deteriorate due to 

- inspectors who have a sort of instinct for what is 
an effective inspection, loosing confidence in 
"personal protection" or "alibi"-type inspections 
and making individual decisions independent of 
management specifications 

- workers motivation for quality being undermined 
by "over-inspection" . 

To cope with this broad spectrum of technical and human aspects 
we decided to review our inspection planning systematically. The method 
developed is based on risk evaluation of work performed and will be des- 
cribed subsequently in this paper. 


RISK EVALUATION 


Risk evaluation is an extension of the various systems of serious- 
ness classification already widely used in industry. 

In addition to the criticality of the consequences of bad workman- 
ship the probability of occurance of such bad workmanship is also taken 
into account. The result of criticality multiplied by its probability 
defines the predicted risk. In the statistics of high frequency events 
such as road accidents this result represents the average expected da- 
mage per event. 

The two factors criticality and probability of bad workmanship de- 
fine an area which is shown in matrix-form in Fig. 1. 

The risk increases from a minimum in the upper left to a maximum 
in the lower right corner. 


Probability of 
bad workmanship 



Figure 1 Risk Matrix 

Based on our conviction that risk evaluation is an excellent ma- 
nagement tool to define priority for the inspection function, all work 
orders have to be systematically evaluated. This means in practice an 
evaluation of each individual task line on each job card, to predict 
the risk involved. As an example all possible maintenance activities 
on a JT8D Jet Engine required a total of 6000 items for evaluation. 




Criticality Classification 


In the past aircraft design has been less complicated, maintenance 
organizations have been smaller and the ground mechanics had a direct 
contact with the flight crew {= user) . A mechanic was often responsible 
for servicing "his" aircraft, for daily maintenance and overhaul, struc- 
ture, components and powerplant. 

Nowadays not only have aircraft become complex systems but also 
the maintenance organizations that go with them. Dozens of handbooks 
contain technical description and organizational procedures. Because a 
lot of specialists lack complete knowledge of the user system it has 
become more difficult to place any emphasis on inspection priorities. 

In an aircraft maintenance organization it therefore seems to be nece- 
ssary to establish a committee for the criticality classification. As 
an example the specialists of the component workshops had to sit to- 
gether with those of the aircraft systems-overhaul and daily mainte- 
nance. In addition, Quality Assurance took part in order to see to it 
that all committees applied the same standard for different types of 
aircraft as well as for electrical/electronic and mechanical systems. 

To achieve the quality objectives required by the management the 
possible results of bad workmanship were classified - similar to Cri- 
tical, Major and Minor - into the following 4 groupes; 

A - No or insignificant consequences 

- No influence on operation or durability 

B - Adversely affecting operation without 
consequential damages 

- Detectable by flight-crew without adversely 
affecting flight safety 

- Detectable by inspection on ground with or 
without consequential damage 

C - Adversely affecting or precluding safe operation 

- Major consequential damages if no corrective 
action is applied 

- Examples: rejected take-off, flight interruption 

D - Hazardous or unsafe condition affecting flight 
attitude or individuals if corrective action 
cannot be applied 

- Endangering human life 
Probability of bad Workmanship 

No statistical material is available for the determination of the 
probability of bad workmanship. Consequently a practical approach to 
that end was therefore made based on the hypothesis that the probabili- 
ty of bad workmanship will diminish the better the resources of men, 
methodes, machines and material become. 

The "human" sources of error are composed partially of such ele- 
ments as lack of basic knowledge, skill and proficiency. It is the re- 
sponsibility of the employer to provide adequate training. In addi- 
tion personality-related attributes such as attentiveness, power of con- 
centration, reliability and care are even more important for the per- 
formance of high quality work. But since these attributes may only be 
improved by great efforts over a long period of time it is better to 
include such criteria in the personnel selection before employment. 



The human influence factor cannot be evaluated in advance because 
the individual performing a specified job is not known. The standard 
of the resource element "men" is simply considered as "good" for the 
probability evaluation. It is then up to the supervisor who assigns a 
particular workorder to assess the individual and adjust the standard 
criteria as necessary. This means John Doe who has not yet completed 
all his training for a new type of aircraft will be classified as "poor" 
instead of "good" by his supervisor. 

Therefore only the technical sources of error are considered in 
the planning stage for predecting the probability of bad workmanship. 

An evaluation of "methods" centers around the degree to which or- 
ders are sufficiently detailed and understandable. In addition the de- 
gree of difficulty of the work process is to be considered. It is ama- 
zing how practical workers seem to be provided with a sort of instinct 
for classifying of their working methods into 4 categories. 

"Machines" include all tooling and equipment required to perform 
the work. The main criteria is the degree of difficulty in its use or 
operating. Nothing can go wrong with a foolproof set-up. It is obvious- 
ly more difficult if measurements, adjustments or switchings are to be 
made. In order to standardize the evaluation a classification of spe- 
cial tools and electrical/electronic test equipment was developed. 

As to "material", its properties will give an indication of the 
degree of difficulty involved in its handling. Titanium for example re- 
quires the adherence to special procedures for the use of machine or 
hand tools. In a similar sense corrosion-, crack- or f atigue-proneness 
will influence the evaluation of maintenance work on aircraft struc- 
tures . 

The evaluation is carried out with the aid of an auxiliary table 
shown in Fig. 2. For practical purposes score values of 2, 4, 8, 16 are 
assigned to the classification and summarized to determine the proba- 
bility of bad workmanship. 


Resources 

Criteria 

2 

Sc 

4 

ore 

8 

16 

Men 

Qualification 

good 




Methodes 

Degree of 
Difficulty 

easy , 

average 

difficult 

very 

difficult 

Machines 

Operating 

Characteristics 

easy 

average 

difficult 

very 

difficult 

Material 

Handling 

Properties 

easy 

average 

difficult 

very 

difficult 



8-10 

Score 

11-20 

Total 

21-40 

41-64 

Probability of 
bad workmanship 

very im- 
probable 

littl e 
probable 

fairly 

probable 

probable 


Figure 2 Table to establish probability of bad workmanship 





Evaluation example: 


Score 


Men 

Method 

Machine 

Material 


good 

difficult 

average 

easy 


2 

8 

4 

2 


Total 16 "little probable" 

If John Doe with his personal qualification "poor" and corespon- 
ding score of 8 is assigned to this job, the total score is 22 and the 
probability of bad workmanship moves up to "fairly probable". 


SPECIFICATION OF INSPECTION FUNCTION 

After the technical experts have determined the risk, it is up to 
the management to decide its acceptable level. Beside the risk/cost 
analysis the difference in public reaction to an event with 100 fata- 
lities (airplane crash) and 100 events with 1 fatality (road accidents) 
has to be considered. This phenomenon is called "risk aversion" in the 
decision theory. 

In SWISSAIR we use the following 3 inspection procedures: 

The basis for achieving high quality work is the concept of self- 
control, which is applied as a general principle to all tasks regard- 
less of their risk classification. That means that each individual has 
to reread his workorder and check that all required work has been per- 
formed, all procedures adhered to. This may even include an actual in- 
spection. Only when this process of mental checking is concluded each 
individual task is properly signed out by the worker who performed it. 

Tasks where bad workmanship may endanger human life or adversely 
affect the safe operation with an at least fair probability must be 
inspected through an independent double check. This inspection must be 
carried out by an appropriately trained individual who has no authority 
to give any workorders for this particular task nor take any part in 
performing it. It is thus a completely independent inspection function. 

There is an intermediate inspection stage for tasks where bad per- 
formance may adversely affect the safe operation with low probability 
or high probability but detectable by the flight crew. This type of 
double check can be carried out by appropriately trained individuals 
who may be member of the team but are not allowed to check their own 
work . 

For practical purposes the specified double checks and independent 
double checks are listed in an inspection catalogue. If necessary this 
catalogue will be revised twice a year by the same team who originally 
established it. 
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These inspections functions are superimposed on the risk evaluatio 
matrix as shown in Fig. 3. With this method a clear relation is estab- 
lished between the evaluated risk and the inspection function. 



Double Check 


Independent 
Double Check 


Selfcontrol 


Figure 3 Specification of the Inspection Function to the Risk 
Matrix based on selfcontrol for each individual task 
with superimposed double checks or independent double 
checks 


CONCLUSION 

RISK EVALUATION is an effective management tool because it pro- 
vides a systematic approach to specifying the inspection function. Key- 
points in the inspection effort are established largely independent of 
individuals. The classification of inspection function is well balanced 
over a whole range of activities because the decisions are permanently 
compared with each other during the evaluation. Even an open discussion 
about risk evaluation and specification of the inspection function wil 
result in a quality improvement. 
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QUALITY CONTROL AND DATA BASE MANAGEMENT SYSTEMS 


Henry L. Lefevre 
Chief, Quality Analysis Branch 
Air Reserve Personnel Center 
Denver, Colorado 80280 


As computer data bases replace conventional paper work and microfilm systems, 
the need for computer data base quality becomes extremely important. Imagine 
your reaction should a computer call you to active duty as a combat fighter only 
to find that you were 60 years old, and retired. Or suppose you were a captain in 
the reserves who had superior ratings on your last five efficiency ratings with 
four commendations and a presidential citation, and you missed being considered for 
promotion because of an error in the computer data base. 

The Quality Analysis Branch at the Air Reserve Personnel Center is dedicated 
to help prevent these horror stories from becoming realities. The main tools at 
our disposal are: 

(1) Clerical checks of Personnel Records vs the computer data base. 

(2) Quality audits of the clerical effort to ensure that quality standards 
are being met. 

(3) Local computer edits to ensure that clerks are not entering impossible 
or improbable data into the data base. 

(4) Central computer, in-depth edits screening data going into the 
master data base. 

(5) System edits or purges to verify the quality of information 
already in the data base. 

(6) Clerical error correction. 

(7) Computerized error correction. 

Data on reserve personnel normally flows into our data base from other 
computers or from other files stored in our computer by other agencies. As an 
example, when an airman serves his active duty tour and transfers to the Reserves, 
his personnel data is transferred from the Active Military Service File to the 
Reserve File. Errors may have been in the file before the transfer, and errors may 
be created by the transfer. 

In order to have a reliable data base, it is necessary for the incoming data 
to be reliable. That is why clerks check the quality, and Quality Analysis checks 
the clerks. Quality Analysis also reviews the need for both clerical checks and 
quality audits on a monthly basis. When a specific data element flowing into the 
Reserve Data Base is shown to be above the quality standard for a 3-month period. 
Quality Analysis recommends that the data element be dropped from the routine 
clerical review process. When the overall quality of clerically reviewed records is 
better than the quality standard, Quality Analysis stops routine audits for the 
Branch involved. Random follow-up audits will be scheduled in 6-12 months, unless 
routine data base audits show that the quality of the data items monitored by that 
Branch has degraded. 

Local computer edits are generally quite superficial, but they do give prompt 
response to the clerk inputting data or corrections via the computer terminal. 
Typical rejections at this point would be for trying to enter a letter where only 
number entries will fit. The simplicity of these edits is due to the limited 
capability of the prescreening hardware. Their greatest value is that they let the 
clerk know of the error immediately, and thereby contribute to the clerk's training. 


©1978 American Society for Quality Control 


The central computer makes more complex edits of the data before updating the 
data base. Typical screens would include requiring that nonexistent codes are not 
listed in the files. Another check would be to ensure that the date of transfer to 
the Reserves is not greater than the current date. 

Manual and computerized system edits are run to ensure that the systems are 
working properly. The manual edits are slow but thorough. As a consequence, they 
are used to audit critical data or to trouble shoot system problems. A typical 
manual edit would be the evaluation promotion data items on full colonels prior to 
a General's Promotion Board. Another typical manual edit would be evaluation of 
the quality of Mobilization data items prior to a mobilization test. Manual edits 
involve reviewing the computer data base information on those data items against 
the Master Personnel Record. Since manual edits on Mobilization Data items average 
over a half hour per record, the sample size is normally small. 

Computerized edits are much cheaper and less time consuming, but the resultant 
data is seldom as accurate. Systems used in the computerized edits are quite 
similar to those used with incoming data. As an example, the edit may require that 
the year a Reservist finished school must not be later than the year of the edit. 
Another example, would be an interrelationship requiring that a Reservist's address 
should not be unknown unless that Reservist's address status is also unknown. 

Computerized edits can be misleading. An edit of when a Reservist was last 
promoted may show 100% quality if the only criteria is that the data base show 
something other than unknowns. Entries claiming that the Reservist was last 
promoted in 1904 or 1984 would both be accepted as good data. The key to having an 
effective computerized edit is to have criteria that detect all impossible data. 
Improbable criteria should also be eliminated where the data is important and where 
computer programming skills and adequate knowledge of the proper screening criteria 
exists. An example might be the permanent grade of an officer. This data item is 
shown numerically where 01 is a second lieutenant, and 03 is a captain. Numbers 
21-24 are reserved for warrant officers. This edit could easily require that this 
data item be between 01 and 10 or between 21 and 24. This screen detects all 
alphabetical or impossible codes for an officer's grade. The edit could also in- 
clude a requirement to spot al 1 generals under 40, all colonels under 35, etc. 

There could be a general under 40 and there could be a colonel under 35, but the 
probability is low enough that their records should be given a thorough manual edit. 

Statistical techniques are quite valuable in detecting probable errors that 
deserve further review. As an example, it is easy to statistically determine the 
average and standard deviation (a~) for the number of retirement points a reservist 
will have after qualifying for retirement at age 60. The record of anyone having 
less than the mean minus 3o~ points or more than the mean plus 3cr points should be 
thoroughly reviewed. It is relatively easy to have the computer screen 80,000 
records against these criteria. 

A correlation of age versus grade could also be used to detect probable errors 
in the data base. 


ERROR CORRECTION 

The importance of a data item has a critical impact on how that item will be 
edited if it is to be edited at all. Reservist address, as an example, is critical. 
If we do not know how to contact that reservist, his other statistics are of little 
value to the system. 

The impact of an error is also important when it comes to the number of 
retirement points accrued. It is more hazardous to shortchange a 20-year Reservist 
who is about to retire than one who is about to complete his obligation to the 
Reserves. There is no doubt that the retirement points will influence the 20-year 
Reservist's retirement pay whereas the short term Reservist may drop out of sight 
as soon as his obligation is met. 
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Other variables having an influence on how or whether a data item should be 
edited include the economic impact of the error (Reference 1), the quality of the 
data item, and the difficulty of correcting the error. Why spend $10.00 correcting 
the data item when the potential savings to either the government or the Reservist 
will never exceed 10tf. This may be a direct saving as in retirement benefits, or it 
may be indirect based on its effect on the Reservist or the mission of the Air 
Reserve Personnel Center. 

A corrective action effort is nonrial ly initiated when the quality of a data 
item drops significantly below 95%. The first question is "Do we have a systems 
error?" System errors are most common with data created or changed by the computer 
when new records flow into the data base. Computer programs have been developed to 
identify the frequency of the error and even the approximate time that it surfaced. 
Time of occurrence is of value to the systems analysts since system errors are 
normally associated with programming changes. A good computer program is not likely 
to go bad unless someone does something to it. In most cases the programmers were 
correcting another error or "improving" the system. By knowing when the error 
started to occur, the programmers will have a good idea on where to look for the 
offending change in the system. 

When the errors are critical, and not correctable by the computer, the slow 
and tedious processes of manually correcting the records becomes necessary. This 
is where the cost vs benefits evaluation comes into play. 

When the errors are minor or computer generated, a computerized approach is 
considered. Methods used by the census bureau (Reference 2) such as cold deck, 
hot deck, and Monte Carlo corrections are of little value to the Reserve Center. 

The census bureau is working with masses of statistical data and is interested in 
statistical conclusions whereas the Air Reserve Center is primarily involved with 
specific data effecting specific individuals. 

Automatic computer corrections are most effective in our records when a systems 
error has been detected. One example was where a specific data element showed 
2 eros where zero was not of an assigned code. The error was generated when data 
came into the local system. In these cases computerized corrections are relatively 
simple. Computer programs can be written to locate all of the zeros in that 
specific data element and change them to the proper code. However, had this not 
been a systems problem, the corrections would be much more complex. We would 
perform extensive manual audits to ensure that all of the zeros should have been 
another single code, and then make the change--if appropriate. If there were more 
than one correct code for the zeros more complex analysis would be required. For 
example, a 6 in the code in question may correlate perfectly with a 37 in another 
data element shown in the same reservist's record. When this occurs, computerized 
corrections are still feasible. 

Other possible applications for computerized data correction (which we do not 
use) include: 

(1) If a month is shown to be over 12, change it to a 6. You can't be 
more than 6 months off. 

(2) If a day is shown as over 31, change it to a 15. You can't be more 
than 16 days off. 

(3) If sex is unknown, call the reservist male since the population 
in the Reserves is primarily male. 

(4) If 2 data items should correlate, but don't, assume that the data 
item with the least number of possible codes is correct. As an 
example favor numeric codes over alphanumeric codes (Reference 3). 


PROBLEMS WITH AUTOMATED CORRECTIONS 


One of the biggest risks in automated data correction is that the corrections 
frequently create errors that are much more difficult to detect than the original 
error. It is easy to spot August 52, 1970 as being in error, but it is much more 
difficult to spot August 15, 1970 as being in error when the correct date is 
August 1, 1970. Chances are about 30 to 1 that a computer generated August 15, 

1970 is not the correct date. In most of our records this type of approximation 
is not acceptable, although it might be perfectly all right for a census analysis. 

A good rule of thumb would be to limit automatic or computerized data 
correction to errors created by the computer, or areas where the resultant errors 
will not cause serious reprecussions . You don't want to insult a woman by assuming 
that she is male, just because bl i nd statistics favor that conclusion. 
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MANAGEMENT'S ROLE IN THE QUALITY ASSURANCE FUNCTION 


Peter M. Wheeler, Director 
Office of Quality Assurance 
Social Security Administration, DHEW 
Baltimore, Maryland 2.1235 


In fiscal year 1978, the Social Security Administration will authorize payment 
of more than 96 billion dollars in cash benefits to more than 35 million people 
receiving retirement, survivors, and disability insurance benefits and supplemental 
security income benefits. Administering programs of such magnitude, affecting so 
many lives, is an awesome responsibility and requires extreme sophistication. 

Being a public program, it is necessarily subject to public scrutiny. Members 
of the Congress and the press and everyone who has ever paid social security taxes 
or who receives or can expect to receive benefits has a personal interest in these 
programs and how they are managed. As you know, this past year there has been 
somewhat more public interest in financing the retirement, survivors and disability 
insurance programs, than in the efficiency with which they are run. Nonetheless, 
the very size of our operation has caused increasing attention to be given to how 
well we are managing our trust and what we can do to improve our management. 

This afternoon, I will outline some of the problems of program measurement 
inherent in SSA. I will explain the 'nature of our programs and their major 
characteristics and objectives. Then I will explain the nature of the system 
we're designing to measure how well we're doing in meeting our objectives and how 
we intend to plan and initiate corrective actions for those areas we find deficient. 

The Social Security Act and amendments to it establish a number of programs 
which have the objectives of providing some protection for retired and disabled 
workers, their families and survivors of deceased workers, protecting aged and 
disabled persons against health care expenses that could otherwise exhaust their 
savings, keeping families together, and giving children the opportunity to grow up 
in health and security. These programs include: retirement and survivors insurance, 
disability insurance, health insurance (better known as Medicare), supplemental 
security income for the aged, blind, and disabled (SSI), unemployment compensation, 

Aid to Families with Dependent Children, and Medical Assistance for the needy 
(better known as Medicaid), among others. The Federal Government operates the 
retirement, survivors, and disability insurance, Medicare, and the SSI programs. 

The other programs are operated by the States with Federal cooperation. 

Today, I will concern myself mainly with the retirement, survivors and disability 
insurance programs, which account for nearly 93% of the projected $96 billion I 
referred to earlier. Payments under these programs are commonly known as social 
security benefits. These benefits ordinarily are paid by check directly to the 
beneficiary, and are thus referred to as "cash benefits" to distinguish them from 
Medicare benefits which are generally paid on a beneficiary's behalf to the person 
or organization that furnished health care services. 

Monthly social security cash benefits can be paid to: a disabled insured worker 
under age 65; a retired insured worker at age 62 or over; and to the spouse or 
dependent child of an insured retired, disabled, or deceased worker under certain 
circumstances. 

By way of example, for November 1977, the retirement and survivors insurance 
program (RSI) paid out nearly $6% billion and the disability insurance program 
(DI) paid about $1 billion. 

A prime objective of the Social Security Administration is to get the right 
check to the right person at the right time. To further this goal, the Office of 
Quality Assurance (OQA) is charged with determining the extent to which SSA policies 
and procedures support that objective. To accomplish this, OQA is designing and will 
administer a sample review system for the RSI and DI programs. This system will be 
similar in many respects to the system we have been operating for SSI since 1974. 
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We are planning on initiating the RSI and DI reviews when the next fiscal year 
starts, in October 1978. 

Our review data will be tabulated and evaluated so that we can report on program 
quality and develop recommendations concerning improvements in program operations. 

The objective of our reporting system is to assure that top management is regularly 
provided with valid and timely information which it needs about program quality. 

As you can well imagine, failure of beneficiaries to receive their checks 
promptly and in the correct amounts can result in considerable hardship. However, 
unlike the situation which prevails in industry, the government does not risk the 
loss of a customer and future sales for lack of quality. For better or worse, the 
beneficiary can't take his business elsewhere. Thus, the loss to the beneficiary 
resulting from substandard government service on which he is forced to rely cannot 
be measured in terms of dollars alone, but must also be considered in terms of 
humanitarian needs. But that's a two-way street. If SSA, for example, doesn't pay 
the right amount at the right time to the right person, it can be the beneficiary's 
fault in circumstances where he or she was required to make a report but failed to 
do so. Thus, the review system which we design must include the capability of 
allocating responsibility for failure to meet the program objective. Without such 
an allocation, we would be faced with error rates with significant hidden components 
for which management would have little hope of effecting reductions. 

For the retirement, survivors and disability insurance programs, we are designing 
four primary samples, each of which covers a six-month period. For cases in payment 
status, 6,000 retirement cases will be selected, and 6,000 disability cases will be 
selected. For cases not in payment status — that is, situations in which benefits are 
suspended, terminated or applications are disallowed 2,000 cases for each six-month 
period will be selected from each of the two programs. Thus, each month, a thousand 
payment cases, and a third that many nonpayment cases, will be randomly chosen from 
each of the two programs. These cases are identified from our Master Beneficiary 
Record. This Record contains a list of all persons receiving benefits, or who have 
received benefits in the recent past. 

The number of cases to be selected for the payment sample was originally designed 
to be large enough to give precision limits of plus or minus one percent at the 
ninety-five percent confidence level, while allowing for some exclusions which will 
not bias the sample. However, this would have required less than 2,000 cases. 

Given the large variety of reasons why payments can be incorrect in our programs, 
we considered it necessary to increase the sample size substantially so that our 
tabulations would distribute the causes for such incorrect payments in such a way 
that trends and concentrations of problems *jould show up clearly. Only in this way 
would we have the basis for initiating special studies for intensive evaluation of 
problems uncovered by our findings or for planning corrective actions. Thus, the 
sample is about 3 times what is expected to be needed to meet reliability specifi- 
cations. Moreover, if our assumptions concerning projected payment discrepancy 
rates are too low — and this is possible because we have no precedent in the RSI or 
DI programs for across-the-rolls estimates of such rates — we will still meet the 
desired reliability criteria. 

Cases will be selected and reviewed on an account basis. This means that the 
family benefit amount based on the earnings record of the individual whose social 
security number is sampled, will be reviewed for accuracy. 

The major indicators of program quality which OQA will measure in its sample 
review system are the accuracy of payments (that is, the proportion and amount of 
dollars overpaid and underpaid) , and the accuracy of negative case decisions which 
include disallowances of new claims and suspensions and terminations of claims 
previously paid. 

Information related to the accuracy of payments and accuracy of nonpayment 
decisions will be categorized in terms of whether a deficiency originates in a process 
controllable by SSA. For example, an administrative, deficiency may result from an 
incorrect arithmetic computation of benefits. An uncontrollable deficiency is defined 
to be one over which SSA had little or no control. Ordinarily, this would occur when 
a beneficiary fails to report a change in circumstances as required. 


Reviews of sampled cases will be of two kinds: desk reviews and field reviews. 
The desk reviews will be conducted in our 10 QA Regional Offices, and the field 
reviews will originate in our 27 field offices, which service the 50 States and the 
District of Columbia. 

When a case is sampled, the claims folder will be requested automatically. RSI 
claims folders are stored in 6 Program Service Centers, located in New York City, 
Philadelphia, Birmingham, Chicago, Kansas City, and San Francisco. Mos.t DI claims 
folders are stored in Baltimore. For both programs, some inactive claims folders 
are stored in Federal Records Centers, but are retrievable when needed. 

The content of the claims folders ordinarily include the orginal applications 
for benefits, evidence to support the applications (such as proofs of age, marriage, 
and death), and a certified copy of the insured individual's record of earnings 
covered by social security. It also includes a history of payments, adjustments, 
suspensions, and termination actions for the claim. In addition to the claims folder, 
the desk review will also require review of a printed summary of the Master Benefi- 
ciary Record and other documentation which may be available in SSA's files. The 
review of these documents will constitute, the desk review, the findings of which will 
be summarized, coded, and input through SSA's nationwide computer system into OQA's 
special data base. 

Concurrently, the printed computer summary will be sent to the OQA field office 
servicing the address of the sampled beneficiary. It will be used as the basis for 
determining whether the criteria are met for initiating contact with the beneficiary 
and how such contact should be made. For example, it may not be cost-effective 

for us to visit a 78 year old retired worker since the visit, per se, could not serve 

to provide entitlement data which we could not obtain from our files. An alternative 
might be to send a letter asking him to verify whether and when he received his 
check for the sample month, and the amount of the check. This would serve to verify 
that our system accurately reflects his status. 

If a field visit is appropriate, usual survey techniques will be employed. That 

is, all beneficiaries entitled on the sampled earnings record will be notified of 

when we plan to visit and requested to have certain information and/or documentation 
available when our representative visits. An acknowledgement notice will be included 
for postage-free response. 

If information collected from the beneficiaries is not sufficiently documented, 
we will undertake contacts with collateral sources of evidence. For example, if 
the beneficiary does not have a copy of his/her marriage certificate, we might contact 
the custodian of the public record to obtain verification of the marriage. 

Upon completion of data collection in the field reviews, the findings will be 
summarized, coded, and input to the central office computer in the same way as 
described for desk reviews. Should findings of the two types of reviews disagree, 
an exception process will alert us to the fact. We believe that the way in which 
the system is being developed will minimize the frequency of such occurrences. But 
when they occur, we will associate the findings from the separate segments of the 
review and undertake a reconciliation. 

We must deal with large numbers of claims folders — millions of pieces of paper — 
whose movement should be minimal to exercise effective controls. We also have to 
keep travel expenses of reviewers down. Thus, we have developed the split review 
approach which has just been described. We recognize that there would te advantages 
to having one person do the complete review, but cost effectiveness issues had to be 
considered, and this became another example of how management plays a role in the 
development and operation of a quality assurance function. 

As noted earlier, our primary samples will be supplemented by special studies. 
These will encompass three broad areas of interest. First, there will he intensive 
studies of areas which our basic findings highlight as problems or potential problems. 
An example might be a situation in which we find that we have an unusually large 
proportion of incorrect payments to children, but no clear delineation 6f the causes. 
Since there are several categories of children's benefits, we could select a fairly 
large, stratified sample as a special study to pin down where the problem is. This 
would be a logical outgrowth of our ongoing primary study findings. In fact we are 


so convinced that the payoff for quality measurement is in corrective action that 
half of the resources available to us have been committed to special studies. 

A second area would be periodic special studies to monitor areas which cannot 
be measured with precision in the primary samples. For example, the nature of the 
social security annual earnings test (that is, the limit on earnings of benefi- 
ciaries under age 72 before they start to lose benefits) is such that it can’t be 
reviewed during the year; it can only be evaluated after the close of the year. Thus, 
for us to measure the effectiveness of both SSA's administration of the test and 
beneficiary compliance with reporting requirements under the test, we will probably 
conduct routine annual special sample studies of it. 

The third type of special study may be characterized as the ad hoc sample. This 
will represent unscheduled reviews into areas which are not normally the focus of our 
quality reviews; one-time studies, for the most part, which do not stem from findings 
of our primary samples. For example, there is a category of individuals age 72 or 
older who receive benefits even though they were not insured under social security. 

This was a temporary provision which, as a practical matter, doesn’t permit additions 
to the rolls, whose average payments are very low, and whose benefits and cost of 
administration are reimbursed in full to the Old-Age and Survivors Insurance Trust 
Fund from the general funds of the Treasury. Because of the small and rapidly 
diminishing size of this group, it is impractical to include them in our primary 
samples. Should we see a special need to evaluate this group, an ad hoc sample study 
would most likely be the best approach. 

The three types of special studies just described emphasize the role of 
management in quality assurance. They illustrate the wide variety of decision making 
opportunities which occur and are needed in the effective deployment of resources 
just to monitor the health of the programs which we administer. 

Until now I have alluded only briefly to corrective action. But the whole focus 
of our system is to develop information on which plans for such action can be formu- 
lated. This is akin to an industrial quality control operation in which, for example, 
ball bearings are measured on a sample basis. If tolerances are exceeded, management 
must determine whether settings of tools must be adjusted or whether the manufacturing 
equipment needs replacement. Careful data collection will show what the appropriate 
remedy will be so that disruptions to both quantity and quality of production are 
minimized. Similarly, findings from our RSI and DI quality review samples, as with our 
presently operating SSI sample, will identify and categorize the nature and sources of 
deficiencies in our programs — with heavy emphasis on where and why overpayments and 
underpayments to beneficiaries are occurring. 

I should note that consideration has been given to the consequences of the 
Social Security Administration not undertaking an across-the-rolls measurement of 
benefit payments. (Currently, we measure only a very small proportion of payments.) 
After all, this new system represents a major commitment of resources — several 
hundred full-time employees, extensive travel, and allocation of sophisticated compu- 
ter equipment which, together, represent a substantial annual investment of funds. 

SSA's top management has concluded that it would be irresponsible to continue our 
operations without such a system. If we don't measure the accuracy with which we 
make all payments, we fail to meet basic management needs. We also fail to perfect 
our capacity to respond to requests for information which we are already receiving 
from the Congress and the press. We recognize that national resources are necessarily 
limited and that we invite avoidable unfavorable criticism by not proceeding quickly 
to both measure and improve the accuracy of payments we make. Moreover, we recognize 
the responsibility to develop a system which will not only be respected internally, 
but will pass the careful scrutiny of the Department’s Audit Agency, the General 
Accounting Office, and others. 

Should we fail to institute our program of measurement, we would obviously be 
unable to take appropriate corrective actions. Hence, we would be unable to meet our 
own objective of furnishing the level of service to the public which constitutes the 
best which can be provided with the funds available. The public and our own 
consciences combine to demand that we strive to achieve that objective with all the 
tools at our command. In SSA, quality assurance is one of those tools. But it 
requires continuous management attention to assure that it serves as an objective 
source of useful information. Without such attention it would, itself, become a 
misleading and wasteful element in our agency's operations. 
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EVALUATING THE QUALITY OF STATISTICS OF INCOME 


Otto Schwartz, Chief, Quality Control Section 
Statistics Division, Internal Revenue Service 
Washington, D. C. 20024 


The recent wave of consumerism has resulted in an ever-increasing awareness of 
product reliability by both the producer and the user. The user demands and has a 
right to be informed about the reliability of an item whether it is a manufactured 
product, a service, or, as in our case, a published statistic of income. Since de- 
fects in statistics are much less likely to be detected by the user than in products 
or services, we are entrusted with a great responsibility to either publish perfect 
statistics or indicate their degree of reliability. 

This paper describes the system we have designed and developed to enable us to 
evaluate the quality of the data contents of the computer file from which the pub- 
lished statistical tabulations are derived. The file is the end result of various 
data processing operations, including the manual abstraction of data from a sample of 
tax returns, transcription, consistency testing, weighting and error resolution. 

A major function of the Statistics Division in the Internal Revenue Service is 
the publication of Statistics of Income (SOI) . This is done basically by abstracting 
information from a sample of the tax returns filed by the taxpayers, transcribing it 
onto tape, subjecting it to consistency testing, processing it through error resolu- 
tion, weighting it, and publishing the resultant data. 

It can readily be understood that quality control in published statistics is more 
difficult to appreciate than in a manufactured product. Quality control in a color TV 
set, for example, can easily be determined by the consumer by such simple questions 
as: Does the picture come on? Is there sound? Is the color satisfactory? Do all the 
different control knobs work? Does the set operate properly for an expected period of 
time? Realizing that a consumer can generally determine the quality and reliability 
of a product once he purchases it and that he will complain and request redress if not 
satisfied, the manufacturer builds in a certain degree of quality and reliability. He 
carefully weighs the costs of loss of good will, refunds, exchanges, and possible re- 
call if there are quality problems against the costs involved in a quality control 
program to prevent such problems. He will then choose an optimum processing plan that 
will in the long run produce the best financial return for his company. Similar 
conditions prevail in the quality control and reliability aspects involving services 
provided to a customer, such as repairs and legal, financial, medical, business and 
government services. 

In published statistics, however, the picture is different. The user of the data 
can often not readily recognize good or poor quality. He cannot, as with a manu- 
factured item, determine if a numerical value works or doesn't work. He cannot tell 
if a million dollar figure is off by a dollar, ten, a hundred, or even a thousand 
dollars. Experience has shown that users will detect format errors in tabular presen- 
tations such as erroneous columnar headings, stub items, etc., and also figures that 
do not fit into a pattern. But generally they will not detect errors in figures 
within a table unless they specifically analyze a certain figure in detail, utilizing 
various other related sources of information, and are very knowledgeable in the field 
dealt with by the item. 

This situation puts us into a very unusual situation. Why have comprehensive 
quality control when the results are often not recognizable? It is a matter of trust 
and confidence by the user. The user, except for the stated sampling error, expects 
the data to be completely reliable and we have an obligation to provide reliable data. 
Further, we should be able to inform the user about the degree of reliability we are 
providing him. 

We have taken many steps to control the quality of the data we publish, empha- 
sizing the areas where errors are most likely to occur and where they have the best 
chance of being corrected properly. However, we are still not able to, with suffi- 
cient confidence, tell the user how reliable the data are that we are providing to him. 
This paper will deal with what we are currently doing to bridge that gap. 
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Following is a brief summary of the processing operations involved in the prepa- 
ration of Statistics of Income. Individual income tax returns (Forms 1040) will be 
used in the example although we compile and publish similar statistics for corpo- 
ration, business, gift, estate and fiduciary returns, some on a yearly and some on a 
more infrequent time basis. 

Through discussions with users of income tax data, such as government agencies 
who use these data for economic and fiscal planning and development, our Division 
determines what information is to be extracted from income tax returns. Instructions 
and procedures are then prepared for the abstraction of the information. This in- 
cludes the preparation of an edit sheet (abstract sheet) , editing instructions to 
direct how to abstract the information, and various other instructions such as sam- 
pling, controlling, quality control, shipping, consistency testing and error resolu- 
tion. 


Of the 130,000,000 tax returns filed during the year, about 360,000 are randomly 
sampled for the various Statistics of Income programs. The Individual sample in- 
volves about 200,000 out of the nearly 90,000,000 tax returns filed. This includes 
business, nonbusiness and farm returns. The 1040 returns are filed at 10 Internal 
Revenue Service Centers throughout the country and this is where the SOI sample is 
also processed. 

Abstracting of tax return data for SOI is performed immediately following the 
revenue processing of the returns. (Revenue processing is the service center review 
for mathematical accuracy and compliance with the Internal Revenue Code.) Utilizing 
our various Internal Revenue Manual instructions, the SOI returns, which have been 
computer-sampled earlier, are pulled, edited, verified and transcribed onto tape. 
Subsequent to this, the on-tape SOI data from all 10 processing centers are forwarded 
to the IRS Data Center in Detroit and subjected to consistency testing. Inconsis- 
tencies are either corrected automatically internally or through manual error 
resolution. Then the data are weighted, consolidated and statistical tables are 
produced, reviewed and published in the form of SOI reports with accompanying de- 
scriptive narrative. 

The quality of the statistics has always played a large role in the SOI process- 
ing operation. However, our main thrust of quality review and quality measurement 
has been in the editing phase. Originally it consisted of 100 percent review of all 
data and it subsequently led to 100 percent review of critical areas of the data, 
lot-by-lot acceptance sampling, and continuous sampling verification. Assuming, and 
we think properly so, that the most critical area of quality emphasis should be when 
the data are first abstracted, we are currently using a fairly comprehensive quality 
control program for this phase. It is a two-pronged operation: one involves an 
inprocess review of the edit sheets by people within the editing unit, utilizing a 
continuous dependent verification plan; the other involves an independent review in 
the Statistics Division of a subsample of the edit sheets prepared in the field. 

Both reviews operate on the same basic premise, that an edit sheet wherein one or 
more items are abstracted incorrectly (excluding errors within designated tolerances) 
is considered defective. 

Edit verification is intended to control the quality. The assignment of a spe- 
cific "i" value (clearance number) and "f" value (sampling frequency after an "i" 
number of consecutive edit sheets have been found to be defect-free) to a specific 
category and type of document (e.g., 1040 Nonbusiness Small) assures that the percent 
defective edit sheets will not exceed a certain "Average Outgoing Quality Limit." 

The independent review of a small subsample of the SOI edit sheets is intended 
to measure the actual outgoing quality of the edit sheets in addition to detecting 
errors caused through inadequate instructions. It provides feedback to the processing 
centers on error data thus allowing for subsequent upgrading and ensuring uniformity 
of instruction utilization among the processing centers. 

It appears evident that the quality of the editing phase of SOI processing is 
fairly well covered. However, the SOI data, as pointed out earlier, is subjected to 
several additional processing stages: transcription onto tape and consistency testing/ 
error resolution. Although quality control measures are applied to these subsequent 
operations they are mainly for review, not for measurement. For example, in tran- 
scription, each edit sheet is retranscribed by a key-verifier and any non-matching 
entries are supposedly resolved. Studies however have indicated that this review 
plan, though seemingly infallible, has allowed some error conditions to remain pre- 
sent or even introduced additional ones. For example, a key-verifier with his power 



to override an original entry can erroneously introduce an error, inadvertently 
retain an original entry error by keying in the same error, or can purposely retain 
an original entry error by arbitrarily failing to correct it. The true extent of 
these error conditions is currently unknown although the service centers in their 
Service Center Quality Review System have established that this problem does exist. 

In the consistency test/error resolution phase of the SOI processing operation, 
errors again may not be detected or may even be introduced. Consistency testing is 
limited in scope to items that lend themselves to this type of check. If an item 
or a combination of items fail a certain test, it generally means that an error con- 
dition is present. During error resolution, corrections are made to one or more 
items to render the data on an edit sheet as a whole consistent. This is often done 
without the benefit of the tax return. Possibilities of incorrect "corrections" are 
thus present. Small studies in this area have borne this out. Again, as with tran- 
scription error, we do not know the extent of this type of error. 

In summary, although a fairly reliable estimate of the data quality is available 
for the editing phase, we recognize that the quality of the final published data may 
not portray the editing quality because of errors introduced and/or corrected in 
subsequent phases of the SOI processing operation. We feel that the user of our data 
has a right to be informed about the quality of the data we are providing to him and 
that we have a responsibility to provide him with this information. Although we 
feel our data are fairly reliable, we want to be able to back it up with statistical 
data . 


To establish the reliability of our published data, we developed and implemented 
a project last year called the SOI Data Reliability Project (or Study). Initially, 
we are implementing the Reliability Project on the three major SOI programs, the 
Individual, Corporation, and Partnership programs. 

Basically, the Reliability Study is an extension of the existing Quality Assur- 
ance procedures. As was pointed out earlier, we currently measure the quality of the 
abstracting by independently reviewing a randomly selected subsample of the returns 
that are edited. This Quality Assurance subsample for the Individual Income tax 
returns involves approximately 2% of the SOI sample. There are three main categories 
of returns included in the SOI program: nonbusiness, business, and farm. The Quality 
Assurance subsample comprises a different percentage of returns from each of these 
categories, determined by the size of the SOI population in each category. 

The evaluation of the final quality, the computer file used to produce tables 
for publication, will utilize the same Quality Assurance subsample and follow it 
through the various post-editing processing stages. Since all of the post-editing 
data are on tape, we will utilize the computer as much as possible to aid us in the 
detection of errors and subsequent evaluation. 

The Reliability Study entails various operations subsequent to the Quality 
Assurance processing. First, we will retain all of the Quality Assurance edit sheets 
and copies of those tax returns for which the corresponding edit sheets were found to 
be in error during Quality Assurance review. In order to determine what happened to 
the edit sheet data subsequent to editing, we have requested two basic computer print- 
outs for each Quality Assurance edit sheet, one following the transcription of edit 
sheet data onto tape and the other following the resolution of errors read out in 
consistency testing. For transcription and consistency-tested documents that differ 
from the Quality Assurance edit sheets, we will obtain copies of the corresponding 
tax returns. We are also requesting several additional computer printouts that will 
facilitate our analysis and evaluation* These include, for both the transcription 
and the consistency-tested files, summary listings for all Quality Assurance records 
of aggregate dollar amounts on a cell-by-cell (item-by-item) basis for all money items 
and aggregate frequency of occurrence counts on a cell-by-cell basis for all code and 
money items. In addition we will match the transcription file against the consist- 
ency-tested file for all Quality Assurance documents and obtain a listing of all 
consistency-tested records that differ on a cell-by-cell comparison of both edit sheet 
code values and dollar amounts (within a tolerance of - $25 for dollar amounts) with 
the matching transcription record. 

Through the utilization of the various computer records and the filed Quality 
Assurance edit sheets and tax returns, we will analyze and evaluate the error data in 
the various stages of SOI processing and eventually determine the reliability of the 
final computer file. We will compare the transcription data to the edit sheet and 
determine the differences. Both the frequency and dollar magnitude of the transcrip- 



tion errors will thus be determined. We will then analyze the mismatched items from 
the comparison of the transcription file with the consistency- tested file. If the 
consistency-tested file contains differences other than errors detected during Quality 
Assurance processing and errors made during transcription, the tax return will be 
consulted to determine if the consistency-tested item is consistent with the tax 
return data. If not, it will be considered a consistency test error. At this stage 
we will determine the error rate of the consistency-tested file. In determining 
whether the consistency-tested item reflects the correct tax return data, proper 
consideration will be given to possibilities of taxpayer errors. Subject matter 
specialists throughout the Division may be consulted in making this determination. 

Once all of this evaluating and analyzing has been performed, we will be able to 
indicate the reliability of the computer file, i.e., the number of SOI edit sheets in 
error, the number of individual cells in error, and the dollar magnitude of the errors 
on a cell-by-cell basis. The prime final error rate of the computer file, the per- 
cent of documents defective, will be determined as follows: 

Percent defective of Quality Assurance edit sheet records 

+ Percent defective of the transcription records (excluding those also QA % 
defective) 

+ Percent defective of the consistency-tested records (excluding those also QA 
or transcription % defective) 

- Percent of records corrected in consistency testing/error resolution 

The reliability data will be weighted based on the Quality Assurance subsample 
sizes of the various SOI return categories and the sample limitations will be indi- 
cated. 

Results of the Reliability Study, in addition to informing us of the reliability 
of the computer file, will show us at which stages errors are introduced during SOI 
processing, how significant they are and if, when, how, and to what degree they are 
corrected in subsequent processing stages. This information will enable us to deter- 
mine what additional if any quality control measures should be applied to the post- 
editing processing of SOI. If the error rates are acceptably low, we will not make 
any substantial changes. If not, we will determine where and how to take corrective 
action. Quality emphasis and resources can be redirected from editing to transcrip- 
tion or to consistency testing/error resolution to achieve the most reliable product 
with the given resources. 
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SPARE PARTS PROVISIONING ANALYSIS FOR POWER PLANTS 


B. W. (Ben) Marguglio, Director 
Quality Assurance - Projects, Engineering & Construction 
Consumers Power Company 


ABSTRACT 

The purpose of this paper is to provide a recommended analytical approach to the 
identification of each spare and replacement part or material and to the determination 
of an appropriate initial procurement quantity for each spare and replacement part or 
material. The paper describes problems which have been encountered in attempts to 
identify and quantify spares and correspondingly provides proposed solutions to these 
problems. The paper also describes each of the factors which should be,. considered in 
the analysis. 


INTRODUCTION 

In performing a spare and replacement parts provisioning analysis for a power 
plant, the analytical process can be divided into four categories — namely, (l) the 
identification of each spare and replacement part or material; (2) based on some 
statistical estimating technique, the determination of a best estimate quantity for 
each of these parts and materials; (3) the adjustment to the best estimate quantity 
based on nonstatistical factors; and, ( 4 ) the determination of quantities for special 
reserves to accommodate unusual circumstances. This paper is organized along the 
lines of these four categories. In addition, a recommended spare and replacement 
parts provisioning analysis form, which follows the steps in the analytical process, 
is provided as Appendix 1 to this paper with the instructions for completing the form 
provided as Appendix 2. 

It must be emphasized that at the time of this writing we have had no conclusive, 
long-term experience with this analytical process. However, we have recognized the 
need for improving the existing spares provisioning process and we have had extensive 
discussions on this recommended approach. We have incorporated this approach into 
our Request For Proposal for our Spare and Replacement Parts Provisioning Program for 
the Midland nuclear plants. It might be appropriate, therefore, if the experience 
with this analytical approach were reported at some future meeting in a year or two 
at which time we should be able to determine its degree of success as well as to 
identify its needed refinements. 

Parts Identification 


The analysis must start with the identification, on a line-item by line-item 
basis, of each spare part, replacement part and bulk material which will be used for 
replenishment. A spare part or a replacement part is one which will perform the same 
function as did the original part. However, a spare part is capable of being installed 
without any physical adjustment, whereas the installation of the replacement part is 
dependent upon some physical adjustment to the part. For a power plant there are 
probably few replacement parts, whereas for other complex products, which are produced 
in quantity and serviced in the field, there may be many replacement parts. An exam- 
ple of a replacement part might be a wing skin panel for an airplane. The panel 
would probably have to be drilled or shimmed or filed to fit in the field when it is 
used to replace the existing skin panel. 

There is no universally established way of assuring that each spare or replace- 
ment part or bulk replenishment material is identified, but an appropriate way would 
be to require that the preparation of a spare and replacement parts or materials list 
be part of the original design work scope and that each drawing be released, with its 
corresponding spare or replacement parts or materials list (different from the "bill 
of materials"). This makes sense in view of the fact that during the design process 
the designer is most intimately familiar with his design and it is during this time 
that the designer must take into account the spares impact on maintenance and main- 
tainability. During the design process the amount of engineering man-hours required 
to establish the spare or replacement parts or materials list is the least. In 
addition, providing the spare and replacement parts or materials list at this time 
©1978 American Society for Quality Control 



significantly minimi zing the risk of having to perform the spares identification, 
quantification and procurement on a crash basis at some belated time. The lists 
should be available with the delivery of the design and not with the delivery of the 
hardware. At that point it could be too late. 

A utility company which utilizes the services of an architectural engineer and 
of NSSS and turbine suppliers, should incorporate requirements similar to those dis- 
cussed above into the contracts with the A/E and principal suppliers, who, in turn, 
should incorporate these requirements into contracts with sub-tier suppliers having 
design responsibility. 

If spare and replacements parts or materials provisioning analysis is accomplished 
at some time after the release of the design other approaches must be taken to assure 
the systematic identification of each spare and replacement part or material. One 
such approach is to provide an indentured drawing list and to work through each draw- 
ing on the indentured list on a system-by- system basis. 

A primary objective in the process of identifying spare and replacement parts 
and materials is to provide the identification of the part or material in terms of 
the ultimate supplier's part number and nomenclature. In the absence of the ultimate 
supplier's part number and nomenclature, the utility may be forced to continue to 
procure spare and replacement parts and materials through an intermediate supplier 
and this may result in delay or additional cost, especially when dealing with 
balance-of- plant items. In addition, the availability of the ultimate supplier's 
part number and nomenclature enables the recognition of parts or materials which are 
duplicated from one system to another. Being able to identify multiple usage items 
further enables the utility to economize. Fewer total multiple usage spares need be 
procured, but one large procurement can be made in lieu of two or more smaller ones. 

For example if System A requires a single spare for vital Part 1 and System B also 
requires a single spare for vital Part 1, as well as System C — it would be recognized 
that two spares, instead of three would suffice, and they be acquired through a single 
order and not three separate orders. Of course, the same applies to the identifica- 
tion and quantification of bulk materials which have multiple applications. 

Statistical Best Estimate 


Without getting into complex statistical applications which are unnecessary for 
quantifying spare and replacement parts or materials for a power plant, the most 
appropriate approach appears to be the most simple as well. For a given item the 
analyst must estimate the expected operating hours for the period in question. Also, 
the estimated failure rate, use rate or depletion rate must be estimated. By dividing 
the former figure by the latter figure, a best estimate of the quantity of items for 
the period is derived. Unfortunately, for power plants there is a lack of adequate 
failure rate data. Nevertheless, the analyst must make the best estimate using the 
practical experience of maintenance engineering personnel. This is better than trying 
to establish a best estimated quantity on the basis of speculation alone. The 
analyst should also recognize that this method of calculating a best estimated 
quantity or a most likely quantity is predicated upon the assumption that the failure 
rate of the item is random or constant throughout the life of the item. This may not 
be the case for items which are susceptible to higher infant mortality failure rates. 

An important point to note is that the determination of failure rates may go 
hand-in-glove with the performance of other design quality analyses such as proba- 
bilistic cause-and-effects analysis, or probabilistic failure modes and effects 
analysis or quantified reliability analysis — any of which may be a requirement of 
the design process. Much of the data is common to these analytical techniques. This 
tends to substantiate the economic justification for the performance of spare and 
replacement parts analysis in conjunction with these other analyses during the design 
phase. 


Adjustments to Statistical Best Estimate 

The best estimate of the quantity of items required for the period in question 
must be adjusted based on the consideration of a number of other factors. Among 
these factors are: the consequence of not having the spare or replacement item when 
needed; the procurement load time for the item; the cost of the item; the repairability 
of the item; the susceptibility of the item to future design change; and the shelf 
life time of the item. Following, in turn, is a brief discussion of each of these 
factors . 
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If the consequence of not having a spare or replacement part or material is 
high, there may he a desire to add to the statistically best estimated quantity of 
spare or replacement parts or materials. Conversely, if the consequence is low, an 
economy may he achieved by ’’fudging*' on the best estimated quantity. 

Similarly, if the procurement lead time for the part or material is long, 
especially if long lead time exists in conjunction with a high consequence of not 
having the part or material, there may be a desire to provide a quantity which is 
higher than the best estimated quantity. 

If the cost of the item is high, but the item is neither of high consequence 
nor of long lead time, an economy may be achieved by again paring the best estimated 
quantity. 

Some items will be spared at the component level rather than at the piece- 
part level. In other words, some spares will be components and not parts. As 
such, these spares, themselves, may be repairable by the replacement of their parts. 
Wien a component spare is, itself, repairable the quantity of component spares may 
be reduced with recognition of the component .repair turnaround time. 

Sometimes costs a,nd schedule expediencies outweigh the need to maintain a certain 
performance level in which case a design may incorporate lesser items than would have 
been the case otherwise. In such cases there may be a desire to retrofit these items 
within improved items at some later time. In such cases, the quantity of spares for 
the lesser item should be reduced in anticipation of the forthcoming improvement 
change. 

Similarly, if an item's shelf life is shorter than the period for which the 
analysis is being made, the quantity of spare or replacement parts or materials should 
be adjusted downward in recognition of the short shelf life of the item. 

The pluses and minuses for each of these factors should be algebraically summed 
to determine the net adjustment to the best estimate quantity. When this adjustment 
is positive, it may be referred to as "required safety stock". 

Special Reserves 

In the case of a power plant, a special reserve may be established to accommodate 
the checkout, preoperational test, hot functional test and start-up test phases during 
which time there may be expected to be more failures than would be the case at a later 
time. This special reserve for test should be identified separately because upon con- 
clusion of the test phases, this reserve is no longer required and the re-procurement 
setpoints should be adjusted downward in recognition of this fact. 

Also, a special reserve should be established for emergencies. If a spare or 
replacement part or material is an absolute necessity to the operation of the power 
plant, a single unit of this item may be established as an emergency spare which 
would not be allowed to be used in any routine test or maintenance activity. The 
emergency spare would be retained especially for use in any case in which it were 
required to preclude a plant outage. Failure to abide by this restriction would 
destroy the purpose of the emergency reserve. 

Summary 

The main steps of the analytical process are: (l) to establish a system by which 
to assure the complete identification of spare and replacement parts or materials; 

(2) to identify the ultimate source of each part or material in terms of the ultimate 
source’s part number and nomenclature; (3) to estimate a most likely quantity of 
spare and replacement parts or materials to be required for the period in question; 

(M to consider a number of factors, other than statistical factors, from which to 
estimate safety stock; (5) to establish special reserves for test; and, ( 6 ) to assure 
that all of this is accomplished in a timely and economical fashion, preferably during 
the design phase. 
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TREND ANALYSIS 


"The Key to Objective Corrective Action" 

Robert G. Burns, Chief Engineer, Quality Systems Division 
Quality Assurance Department, Stone & Webster Engineering Corporation 

Boston, Massachusetts 


This paper will describe the evolution of a quality performance trend 
analysis system developed specifically for application to large construc- 
tion projects. This system adapts the techniques developed to analyze 
quality trends in the repetitive high-volume manufacturi ng environment 
and applies them to large nuclear power projects. If properly developed, 
a trend analysis system can provide a welcome relief from the most 
common action taken in the nuclear industry today: the "fire fight" or 
"witch hunt". Our current preoccupation with rushing from crisis-to- 
crisis defeats effective correcti ve/pre venti ve action which must be a 
carefully thought out management process. 


Stone & Webster has developed an on-line system which measures quality 
performance by analysis of returned inspection data. As it now exists, 
the program is the composite result of the following four distinct 
phases which will be discussed further. 


0 

PHASE 

I 

Quality Organization 

0 

PHASE 

II 

Program Development 

0 

PHASE 

III - 

Data Analysis 

0 

PHASE 

IV 

Corrective Action 


Phase I of this program, the structuring of our Quality Assurance 
Department, took place over a three-year period and resulted in the 
organization shown in Figure (1). The features of this organization 
are: the establishment of a design control function within the 

Engineering Department designated as the Engineering Assurance Division 
and the creation of a Quality Assurance Department independent of the 
Construction Department. The Quality Assurance Department continued to 
evolve until Phase I was completed by a reorganization on March 19, 1973, 
which resulted in the establishment of a Quality Systems Division and a 
Department Services Group. 


Quality Systems Division, which is responsible for developing a 
standardized program with related performance measurement systems, 
became the quality engineering branch of the Quality Assurance Depart- 
ment. 
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Phase II was immediately initiated concurrent with the establishment 
of the Quality Systems Division. We considered the adverse impact of 
operating in a non-standard mode and its relation to trend analysis. 

The need for each project to have a totally unique system was serving 
to divide and conquer our efforts to establish a sound trend analysis 
system. The effectiveness of performance measurement is, in my judgement, 
directly proportional to our ability to establish standard ODe'rations 
and to adjust these operations in response to feedback data. Similarly, 
in non-standard procedural systems, the error rate (conflicts, omissions 
and additions) increases in proportion to the degree of projecti zati on . 
Development of a standardized procedural system, aDDlicable to multiple 
projects, took approximately three years and is depicted graphi cal ly i n 
Fi g u re ( 2 ) . 


Key Phase II program features are as follows: 

1. Procedures are standard and relate directly to 
engineering specifications. 

2. Detailed attribute lists are generated to accompany 
each procedure. 

3. A standard report format, the Inspection Report (IR), 
see Figure (3), is an integral portion of the system! 

4. Inspection Planning is no longer an optional effort but 
rather an identified duty. 


A key decision was made, in conjunction with Item 3 above, to count 
the "goods" as well as the "bads". This may sound quite simplistic 
in retrospect, but it is the key element in our quality performance 
measurement program. How often have you seen systems that only measure 
ll bads-" while ignoring the process averages? The NRC, ASME and others 
quite commonly, and with some justification, use this technique. 


In order to provide even further definition, we segregate deficient 
conditions of a simple or minor nature from those of a more, complex 
nature. This approach identifies all items found by an inspector 
to be deficient as an unsatisfactory (UNSAT) item or condition. If 
this UNSAT is resolved by the seller (manufacturer) or the site 
constructor, it ends there. However, if the item must be referred 
to project engineering for disposition, the UNSAT is closed and the 
item reappears on a Nonconformance and Disposition Report (N&D) as a 
nonconformance. This two loop system places quality corrective action 
at the lowest most responsive level; also, it allows the data base to 
identify the following conditions: 


1. Avoidance of effective corrective action by 
sellers or cons tructors . 

2. Dependence or costly referral of all matters 
to project engineering. 

3. Failure of project engineering to produce 
clearcut engineering di recti on in specifications 
and drawings. 


A graphical depiction of the two loop system may be seen in Figure (4). 



Phase III, Data Analysis, utilizes our large scale in-house computer 
capability. Data is collected in two forms: first, as hard copy 
to be coded or keypunched for entry into an IBM 370/168; and secondly, 
by the most desirable direct entry from job sites. 


Data, once entered, is stored, retrieved and manipulated to make use of 
the data base. We currently produce a number of status reports, 
measurement and test equipment reports and quality performance reports. 
The Keyword Statistics Report, shown in Figure (5), is one example of 
a quality performance report. 


This report is issued on a project basis with the capability of assembl- 
ing data into an across projects summary to determine overall process 
averages. This report compiles and displays quality data in the 
following order: 


o Commodity (Pumps, Rebar, etc.) 
o Inspection Activity (Shop, Field, Final, etc.) 
o Total Population 
o Number Inspected 
o Number Unsatisfactory 
o Apparent Percent Defective (APD) 


The APD is determined by the ratio of the Number Unsat to the 
Number Inspected, times 100. We have adopted this term to avoid 
confusing it with the true percent defective (TPD), which may be 
determined by 100 percent inspection of the population to determine 
actual performance without resorting to statistical inference. 


In addition to raw statistical material, we provide the user with a 
listing of the worst three attributes in that month and the previous 
two months rates for those attributes. 


The ultimate purpose of this report is to take corrective action, based 
on our overall and individual project performance. By digesting the 
mass of data on a project (see Figure (6) ) we can bring problems into 
focus for both supervisory and top level management action. 


At this juncture, I would like to point out some pitfalls to be avoided 
based on our development experience. 


o Do not assume an overly optimistic schedule in a 

standardized program. Your salesmanship and marketing 
ability in this matter, both internally and externally, 
will be equally as important as technical competence. 

o Forms that work in a manual mode probably would not 
work efficiently in the machine mode without expensive 
key punch or coding buffer. We optimized our form to 
suit machine operation, allowing direct data entry by a 
clerical person. 
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o Do not assume people will understand or easily relate to 
such systems. Plan to accompany all developments with 
vigorous indoctrination and training. 

o Do not assume your lofty motives are well understood by 
others. Get a bulletin and person-to-person contact 
campaign going to avoid surprises to system users. 


I do not think we can or should eliminate corrective action on "gut- 
f ee l"--that is always a management prerogative; but, we have provided 
an objective measurement tool that, hopefully, will allow more timely 
and effective objective corrective action. 
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COMPANY ORGANIZATION FOR QUALITY ASSURANCE 
STONE & WEBSTER ENGINEERING CORPORATION 


FIGURE 1 





















PROCEDURAL SYSTEM VS ORGANIZATION 


FIGURE 2 
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STONE 6 WEBSTER ENGINEERING CORPORATION 


FIGURE 5 



CORRECTIVE ACTION OVERVIEW 


FIGURE 6 





POWER GENERATION RELIABILITY 
THROUGH THE BACK DOOR 

Presented By 
Jack E. Vessely 
Manager, Quality Assurance 
Florida Power & Light Company 


It is true that money talks and therefore power plant availability will be 
having a loud voice in almost everyone's future. Availability, being one of the 
measurements of reliability, will in general become the focus of much attention as 
greater emphasis is placed on getting the most energy for the least cost. State 
and Federal legislation has been enacted and more being proposed, focusing on reli- 
ability. For instance, the Michigan Public Service Commission has established an 
incentive/penalty program to encourage Detroit Edison to improve its plant avail- 
ability - offering the utility incentives above its allowed 13.5% return-on-equity 
if it can push system plant availability above the 85.1% level, or penalizing 
Detroit Edison if system plant availability falls below the 70% mark. The plant- 
availability-incentive provision was contained in DE's rate order dated May 27, 1977, 
and is believed to be the first time that such an incentive plan has been adopted in 
a major rate case by any state regulatory commission. Some of this legislation is 
enlightened and some is not, but whatever the cause or eventual effect, it is here 
and will continue to proliferate in the days to come. 

One of the main reasons for improving availability comes down to a monetary 
concern. The major contributing factor is that power generation facilities cost so 
much to construct these days. To date much of our generation availability was 
achieved by having "X" percent of peak demand in surplus on standby. In 1967 the 
estimated investment cost for a nuclear plant was approximately $135/KWe, with a 
5 year service date. In 1973 the cost was $520/KWe and up over $1100/KWe by 1976, 
an increase of over 800% in nine years. With these types of construction costs you 
don't want to be building any more units than you absolutely need to have. Increases 
in individual plant availability have a large effect on when new plants are built or 
deferred. 

The next major contributing factor related to availability is difference in 
operating costs between plants. Once a nuclear plant is built the cost differential 
of fuel involved in generating electricity, from oil or coal, rims at about $20/MWhr. 
which comes out to $350,000/day for a 700 MW nuclear unit. 

The question I wish to address today is how we at Florida Power & Light are 
applying Quality Assurance techniques and other management systems to improve the 
reliability of existing nuclear plants. Suggestions, plans, recommendations and 
actions in this area abound. Many times those are based on the more visible aspects, 
the direct causes of forced outages. These are the items which get publicity and 
therefore become the subject of legislation. It is easy to read an article in the 
paper and then react by saying that the problem should not happen and then take 
action to see that it won't happen again. This works real well in the short range 
since the most obvious causes of failures usually effect availability. Things such 
as faulty valves and pumps in plants and transformers in distribution are common 
subjects of extensive reliability investigations. 

A study was conducted for the Federal Energy Administration/Office of Energy 
Conversion by the Mechanics Research Inc. To identify and assess underlying causes 
of unreliability, our Turkey Point Plants were among the plants included in the study. 
Part of the study conclusion highlighted Turkey Point's ability to identify, resolve 
and implement corrective action. 

- Availability of special problem solving staff at the plant 
dedicated to the resolution of identified problems and 
recommend means of improving plant performances. 
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- Plant management authority and willingness to expedite. 

- Training of plant personnel in trouble shooting. 

- Economic trade-off analysis and management techniques to 
improve their problem solving capabilities. 

- The higher level of instrumentation that provides useful 
pertinent data to help identify problems more efficiently. 

At the present time most efforts are being applied in these areas. In achiev- 
ing the 70 to 80 % availability factors they seem to be meeting the needs, i.e., 
fighting the f.ires . However, in the future, to achieve 80 to 95% availability fac- 
tors, we will need more sophisticated systems applications. The following examples 
show how short-range and long-range problems in availability are being addressed by 
Florida Power & Light. 

At the present time none of the present FPL power generation plants have been 
purchased with specific reliability requirements, incentives for reliable design, or 
life cycle cost considerations identified in the procurement documents. Utilities 
purchased what the manufacturer had to offer on a turnkey basis, just like you buy a 
catalog item off the shelf. With that start in reliability the only thing you can do 
is to identify what you got, using performance to improve your reliability through 
failure history, thereby, the back door approach. And this is where Quality Assur- 
ance plays such a major role in the overall reliability system. This sounds like old 
stuff. You say this to utility operations or the engineering departments and they 
say that is exactly what we are doing, but are they doing it in a planned and orga- 
nized manner or by fighting fires as they come up? 

For the benefit of those who are not familiar with performance measurements of 
power generation plants, there is no single index which adequately reflects genera- 
ting unit reliability. Three separate indices are used; capacity factor, availabil- 
ity factor, and forced outage rate. All three indices must be considered. An orga- 
nized approach to reliability as a whole must address and assess equipment, personnel 
and environmental factors. A method must be in place for determining the effect of 
individual actions and reactions on other components and the entire plant. Figure 1 
shows the typical relationship between reliability indices and primary influences. 
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TYPICAL RELATIONSHIPS BETWEEN RELIABILITY INDICES AND PRIMARY INFLUENCES 


FIGURE 1. 






At the present time we see many examples in the utility industry where relia- 
bility elements are being applied in various forms with various levels of sophistica- 
tion to affect availability. Often these applications are made to correct specific 
existing problems without direct concern as to overall system reliability and do the 
job very effectively in that context. We at FPL are in the process of developing a 
power generation reliability system to be tailored to the existing structure so that 
the needs of the company be addressed. Also, problem areas are being investigated 
in both the quick fix, get the plant on line, and in a manner to improve the life 
cycle cost, forced outaged and availability factors of reliability. 

The key or backbone to the back door approach to reliability is a good failure 
recording and reporting system. Four years ago the Power Resources Department of 
FPL developed a good failure recording and reporting system, a more long-range look 
at availability, called the Generation Equipment Management System (GEMS). This 
system addresses the problem of power plant outages, and is a good example of how a 
system is established to deal with a direct need through standardizing existing in- 
formation into a usable form for big-picture application. The stated purpose of 
GEMS is to standardize the accumulating and reporting of power plant operation and 
equipment outages. 

The system is established as follows. First let's look at input. Determina- 
tion that a job needs to be done comes from three sources. The biggest source is 
the Plant Work Order, which can be generated by anyone who recognizes an equipment 
or area problem. A second source is planning meetings of supervisory personnel who 
decide what jobs should be performed. Finally, there is a preventive maintenance 
schedule which is reviewed monthly and additions, deletions or frequency changes 
may be made. 

From each of these sources a Plant Work Order is generated which becomes the 
key to the reporting system. This form is filled out by anyone requesting work to 
be done and is reviewed by a foreman and supervisor. From there, a planner reviews 
the work, orders any necessary parts or equipment and schedules the operation. All 
work is then assigned on this basis on a daily schedule. The point of interest here 
is the fact that pertinent information from each Plant Work Order is coded for com- 
puter use in various applications. The following information is used. First there 
are code identifiers for plant, unit and the specific equipment involved. Next is 
the major code which identifies the condition which required that the unit or compo- 
nent be taken out of service or run at reduced capacity. Code sheets are regularly 
sent to plants to provide updated information as to potential problem causes. The 
manufacturer is then coded in or the fact that an outage was due to operator error. 
Very important to the overall system is the outage code which classifies an outage 
into forced, maintenance, overhaul, reserve, forced partial, scheduled partial, or 
preventive maintenance curtailing. Another important entry is the action code, which 
describes what is done to retain the unit to normal service. There is also a section 
for outage start date, outage end date, curtailed MW, manhours spent in repair, ma- 
terial cost, and contractor costs. 

All this information is then available as output in various forms. First is 
the summary report. Besides the information on each unit described previously, there 
is effective outage information which accounts for the fact that several repairs can 
be performed in one outage. Also included is available hours, outage hours, opera- 
ting availability, reliability, outage rate, forced outage rate and ratio, and 
scheduled outage rate’. A section also covers various capacity factors for the system. 
Another report lists outage-related work that occurs within a time period, with occur- 
rences grouped by related outages and is available for one or more plants. In addi- 
tion to these standard reports there is a GEMS special report which can search the 
entire master file (all Plant Work Orders ever submitted) and extract specific in- 
formation in a variety of fields including plant, unit, equipment, major cause of 
outage, manufacturer, outage cause, and other identifications, as well as within 
maximum/minimum manhour requirements. 

The uses of this failure information to the overall availability picture are 
numerous. Among the present uses are determining principal causes of outage, 
reliability/availability statistics for future planning, unit reliability factors, 
operations and maintenance program improvements, new equipment purchasing guidelines, 
equipment maintenance history, provisions of operating information to comprehensive 
pools such as Edison Electric Institute. 



Our next step in using the GEMS data is to determine the top 10 contributors to 
systems unreliability in each of the following design categories of plants: 

o Nuclear Plant - Westinghouse Design - 3 loop 
Turkey Point Units 3 & 4 

o Nuclear Plant - Combustion Engineering Design - 2 loop 
St. Lucie Unit 1 

o 400 MW Fossil - Ebasco Design 
Turkey Point Units 1 & 2 
Cape Canaveral Units 1 & 2 
Port Everglades Units 3 & 4 
Fort Meyers Unit 2 

o 400 MW Fossil - Mid-Valley Design 
Sanford Units 3 & 4 

o 850 MW Fossil Plant 
Manatee Unit 1 

The intent is to focus the attention on the most active causes and work on them in a 
planned, organized, cost effective manner, rather than a forced outage situation. 

As you can easily see, this system is capable of taking existing data, putting 
it into useful form and giving useful information as to how short-term decisions can 
affect long-run performance. In this context, availability is becoming' a solid 
management tool for the industry. 

Finally, reliability is being applied to programs involving systems applications. 
One problem facing utilities today is the decision whether to build new generating 
facilities or defer them and if a plant is to be built, what type unit and how large. 
This process becomes more complicated as growth patterns fluctuate more rapidly, as 
production costs increase and as diversification opens up new avenues with more 
opportunities and more choices to be made. 

Historically, the calculation of production costs represented an area where much 
estimation had to be made. Planning studies were inaccurate because the input relied 
on approximations and the output was affected to that extent. One of the major vari- 
ables involved in any estimation of this type is reliability through the forced out- 
age rate. This variable is subject to a number of parameters including type of plant 
and length of time in service with new plants going through numerous minor problems 
and older plants having more unexpected breakdowns. For a variety of reasons these 
conventional methods consistently underestimate the utilization of high cost peaking 
units, and consequently underestimate the real cost of production. 

Our approach to this problem is through PROCOS, designated a production cost 
program, which employs probabilistic techniques to obtain accurate predictions of 
unit loadings. This system basically tells what units should operate at what times 
to give most efficient operation. To do this a load shape is established for a given 
month. Then, a determination is made as to how to have units on line to meet that 
load shape. The basic variables involved here are plant maintenance schedules, heat 
rates, load points (minimum, middle and maximum for each plant) and forced outage 
rate . 


Of interest here is the forced outage rate which is defined as the probability 
that a unit would have to be brought off the line for maintenance before the next 
weekend. That is, the maintenance could not be delayed beyond that time. This 
rate is an input into the PROCOS program and utilizes much of the existing avail- 
ability data to arrive at solid estimates of forced outages. As presently operated 
the rate for each plant comes from three sources. Edison Electric Institute pub- 
lishes data by unit class (such as 100 MW range) and type (oil, coal, etc.). Other 
utilities and industry information is also relied upon. The major input, however, 
is historical data. This comes from a variety of sources including maturity statis- 
tics for particular plants and the GEMS system. In this manner forced outage rates 
are established for each plant and this information is used to come up with a graph- 
ical representation known as a system loss-of- load-probability , (the likelihood that 
the system will not be able to meet the load demand) and is a primary factor in 
determining which plants should be operated to minimize operating costs of the system. 



The PROCOS program can thus be seen as a means by which availability is applied 
on a system basis to greatly reduce generating costs. In this way we are helping 
meet the challenges of today through effective analysis and planning. 

In summary, these are examples of how the management of one utility deals with 
the problem of generation system availability. A number of different approaches are 
currently being used including correction of existing day-to-day problems, tailoring 
of system approaches to the existing utility structure, and setting up long-range 
systems for planning purposes. 

The situation still has a ways to go in the days ahead. Among the areas yet to 
be established are failure indices at the total system level based on availability 
at the point of use including generation, transmission, and distribution. The 
composite performance required can then be used to develop quantitative worth mea- 
surements reflecting actual economic values of the quality of electric service to 
the customer. 
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QUALITY SYSTEMS AUDIT 


Ralph I. Levine 

Manager, Corporate Quality Assurance 
The Gillette Company 
Boston, MA 02199 


I. The Quality Systems Audit is a management concept that can be 
utilized in a variety of ways, at different management levels. Since 
the purpose of this paper is to reflect principles and methodology of 
an audit based on my experiences, it will be presented based on my 
specific circumstances. Hopefully, it will be useful in a broad sense 
to you who may have similar or different needs in the audit area. 

II. There have been many articles published on this, or similar 
subjects, with emphasis on what to audit and detailed checklists. 
Although I intend to expand the details to some degree, I will also 
try to highlight the practical complexities of an audit from an over*- 
all business standpoint, rather than just the philosophical Quality 
Control outlook. Additionally, I will attempt to highlight the how-to 
of any audit. 

In my present capacity, I have the responsibility for performing 
Quality Systems Audits on international and domestic operations. The 
purpose of these audits is to help the local operations in pinpointing 
quality-related weaknesses and strengths within that operation and to 
provide corporate management with an assessment of each operation's 
quality performance, thus leading to appropriate improvement programs. 
Now that the stage has been set, the following is a list of princi- 
ples which may be considered as guides in performing audits: 

A. Forewarn of upcoming audits. 

B. Estimate the length of the audit. 

C. If the audit is in a non-English speaking country, 

arrange for interpretation, as necessary. 

D. Establish the objectives of the audit. 

E. Review the background of the operation to be audited. 

F. Once on-site, outline the schedule of events with local 

management . 

G. Record results during the audit. 

H. Summarize with Quality Control Manager. 

I. Summarize with local senior management. 

J. Issue a timely report, including strengths, weaknesses 

and auditor recommendations. 

K. Follow-up. 

These principles, having been derived empirically, need to be 
expanded in order to understand the audit procedure: 

A. Forewarning 

As a target, try to notify local management of an upcoming 
audit at least two (2) months in advance. The scheduling of an 
audit should be a mutual agreement. Springing a surprise audit 
upon an operation will not buy you anything except resentment from 
that operation's management and future difficulties. Remember, 
you aren't trying to trap the Quality Control department, you are 
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trying to help them reach a common objective, be they an operation 
within your own company, or within some other organization. Further-' 
more, if you don't have a common objective with the operation, your 
auditing will most likely be fruitless. 

B. Estimate Length of Audit 


This is not only a matter of courtesy, but allows the local 
management to plan for availability of personnel. Keep in mind how 
you would like to be treated if the roles were reversed. 

C. Translators 


This is self-explanatory. I would caution, however, that if 
you do get involved with facilities where English is not the mother 
language, be certain that the translator is truly fluent in English 
and not simply the one in the plant with the best English, for that 
might be far from adequate. 

D. Establish the Objectives of the Audit 

This is a key element in conducting a successful Quality Sys- 
tems Audit. It entails answering three questions: 

1. Why am I doing the audit? 

2. What am I going to audit? 

3 . What is the reference scale to be used in rating the 
Quality System? 

1 • Why am I doing the audit ? 

The first question has a variety of answers, depending upon 
the circumstances; be it an operation within your organization, a 
source of supply, etc. Regardless of the circumstances, however, 
the answer (s) should include provisions for: a) appropriate im- 
provement programs based on whatever weaknesses are identified, b) 
identifying those responsible for implementing the programs and c) 
a system for monitoring progress. Without these ingredients, the 
audit will have little value. 

2 . What am I going to audit? 

The second question contains the detail of audit, itself. I 
have chosen a list of key elements in the make-up of a Total Quality 
System (a term coined by A. V. Fiegenbaum) with important consti- 
tuents highlighted within each element: 

A TOTAL QUALITY SYSTEM 

1) Management of Quality 

Quality Control should participate as part of the management 
team. Principal functions include but are not limited to; 

a. Developing quality policies/procedures. 

b. Reviewing consumer use tests, purchaser studies, competitive 
and other field product tests. 

c. Improving product quality. 



d. Monitoring and reducing quality costs. 

e. Training and developing personnel. 

f. Analyzing consumer comments and taking corrective action, as 
applicable . 

g. Establishing attainable quality goals and objectives. 

h. Conducting quality self-evaluation, i.e., auditing own perform- 
ance. 

i. Establishing and implementing product and process control plans. 

j. Organizational efficiency. 

2 ) New/changed Product, Process, and Package Control 

Control of new/changed products should consider the designing, 
reviewing, and/or establishing of the following, as applicable: 

a. calendar for planned new product introductions. 

b. Design review. 

c. Consumer use test. 

d. Specifications; performance, safety, and visual standards; 
classification of defects; and test methods. 

e. Test equipment. 

f. Product life and reliability studies. 

g. Written inspection plan for manufacture of the product. 

h. Test data showing conformance of initial product to specifi- 
cations . 

3) Purchased Materials 

The quality system should provide for procurement of acceptable 
purchased materials and services by means of: 

a. Proper selection of vendors regarding quality. 

b. Issuance to vendors of appropriate specifications, test methods, 
gages, and mutually acceptable AQL's. 

c. Inspection, analysis, and control of purchased material per the 
written quality plan. 

d. Assessment and review of vendors’ performance. 

e. Quality assistance to vendors, as needed. 

4) Control during Manufacture 

In-process control techniques should be established so that 
finished product meets specifications, while minimizing the overall 
cost of quality on the manufacturing floor, by providing for: 



a. Target values of key characteristics. 

b. Acceptance limits of variability about these target values. 

c. Methods of measurement. 

d. Inspection, analysis, and control of material during manufacture 
per the written quality plan. 

e. Information feedback for process correction. 

f. Disposition procedures for non-conforming product. 

Products contract-filled or packaged should be produced in 
accordance with specifications and quality levels. 

5) Control of Outgoing Quality 


An inspection plan for finished product and packaging should be 
established so that product complies with applicable quality require- 
ments by providing for: 

a. Characteristics to be controlled and the degree of compliance 
necessary. 

b. Type of inspection necessary (100% sorting, sampling, etc.) 

c. Inspection, analysis, and control of finished product and pack- 
aging per the written quality plan. 

d. Data collection and feedback for periodic review. 

6 ) Special Studies 

Special studies should be used to support all aspects of total 
quality control, with particular emphasis placed upon improving 
quality and reducing costs by means of such techniques as: 

a. Process capability studies. 

b. Defect prevention studies. 

c. Quality cost analysis and reduction. 

d. Specification review and modification. 

e. Gage and instrument qualification. 

7) Gaging & Instrumentation Control 

Control of all gages and instruments should be exercised so 
that data generated is reliable and truly measures the product or 
process to which it applies. This system should cover all gages and 
instruments — both in-line and off-line — with respect to: 

a. Design. 

b. Set-up and operation. 

c. Updating maintenance and repair. 



8) Documentation Control 


All quality systems should be documented and should provide for 
timely issuing and updating of: 

a. Specifications. 

b. Sampling plans, inspection instructions, and test methods. 

c. Gage/in st rumen t drawings and their associated instructions. 

The documentation procedure should be fully integrated with 
the procedures used by other functions. 

9) Quality Information Feedback 

A Quality information feedback system should be set up to pro- 
vide means for data collection, analysis and reporting on all phases 
of quality and to form the basis for taking appropriate corrective 
action. The type of quality information includes, but is not re- 
stricted to, the following: 

a. Quality levels — purchased materials, during manufacture, and 
finished product. 

b. Quality Costs. 

c. Consumer comments. 

d. Competitive quality levels. 

10) Materials Identification and Control 

A system for materials identification and control should be 
established for: 

a. identifying and controlling materials from incoming through 
finished goods inventory regarding their quality status. 

b. Allowing for required traceability. 

11) Consumer Comments 

A system should be set up, in order to be cognizant of and 
responsive to consumer comments, by providing for: 

a. An orderly and systematic record, evaluation, and control. 

b. Affirmative and effective corrective action. 

c. Prompt and full response to consumer inquiries and service. 

12) Personnel Training and Development 

This approach should consider: 

a. Appropriate orientation or training for those people directly 
assigned to quality responsibility and orientation to those 
whose job impinges on quality. 

b. Training of back-up personnel. 


13 ) Quality Self-Evaluation 


Each Operation or Subsidiary should evaluate the effectiveness 
of its own quality effort. 

a. Evaluation of outgoing product quality, independent of routine 
inspection . 

b. Evaluation of all quality activities — those directly assigned to 
the quality organization as well as others — should cover: 

(1) Product and process. 

(2) Gaging and instrumentation. 

(3) Stores and distribution systems for raw materials and 
finished goods. 

c. Quality self-evaluation should be carried out according to a 
planned schedule with the participation and/or support of the 
Factory Manager, the Quality Manager, and the managers of other 
factory line functions. 

It is these elements plus the interaction of the Quality Con- 
trol group and other functions such as Marketing, R&D, Purchasing, 
Distribution, Production, Industrial Engineering, etc. that are to 
be audited. 

3 . What is the Reference Scale ? 

The third question is the most complex. Those aspects of 
quality in terms of such indices as outgoing product quality levels, 
may be easily assessed if there are established numerical targets 
and limits. The difficulty comes when you try to score such things 
as: a) management ability, b) extent of follow-through within each 

element, c) contribution to the total manufacturing effort, d) effec- 
tiveness of training, e) appropriateness of the goals being set, etc. 
The approach you may consider, and one I've found successful, is 
similar to that of Quality Costs, as depicted in Dr. Juran's Handbook. 
The Quality Cost tool is used primarily to pinpoint areas which offer 
maximum opportunity for improvement. We are sensitive to the risks 
associated with comparing Quality Costs to indices such as sales, 
number of people, production levels, etc. These can be too arbitrary 
and may lead you down the wrong path. 

Much the same approach can be applied to auditing results, i.e., 
use the auditing information to pinpoint those areas which need 
improvement while maintaining the strengths within the organization, 
avoiding the use of arbitrary scales! Therefore, my answer to the 
question "What is the reference scale to be used in rating the 
Quality System?" is to let the list of weaknesses, recommended im- 
provements, and identified strengths speak for themselves. 

Review Background of The Facility 

Every operation may not have the same needs in their quality con- 
trol activities, nor the same emphasis on the various elements, as 
other operations . Each operation should tailor its quality control 
efforts to fit their business needs and so the auditor should have 
an understanding of the other business aspects to help ensure those 
needs are understood. The auditor should review such aspects as: 
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• Financial Status 

• Product Lines 

• Process Types 

• Other Quality Indices 

• Past Audits, if any 

• Local Applicable Laws 

• Local Labor Conditions 

• Market Atmosphere, i.e.. Economy, Competition, etc. 

F. On-Site Schedule 

It is advisable to initially meet with the Quality Manager in 
order to explain the sequence of events in the audit, including the 
people and places you plan to visit. This allows sufficient time 
for personnel to adjust their schedules. This itinerary may and 
should also include functions within the operation, such as Material 
Control, Purchasing, Marketing, etc. It is also advisable to leave 
some unplanned time in case you have to modify the schedule or back- 
track, depending upon your findings. 

G • Recordin g Results 

As you conduct the audit, i.e., reviewing inspection and cali- 
bration data, reviewing documentation, asking questions, requesting 
demonstrations, following the flow of material, etc. there is a need 
to record the findings. This can be done in many obvious ways: 
by memory, a note pad, portable tape recorder, taking copies of 
documents, samples of product and/or photographs. You may think of 
others. My own preference is using a note pad and requesting copies 
of documents. I would be especially apprehensive of using a tape 
recorder as it has a tendency to inhibit people. At this time, two 
items are worthy of comment: 

• It is my opinion that the best auditor is no match for an 
operation that wants to stymie an auditor; therefore, to be 

successful, you must try to instill a feeling of mutual 
respect and confidence. 

• One item usually associated with Quality Audits is a check- 
list. I must admit my bias against them for they tend to 
require so much attention of the auditor that the opportu- 
nity to observe and interpret is hindered. 

To help better understand the detailed activity of an audit, 
the following is an example of how you might delve into one small 
aspect of a quality system. For simplicity, I have chosen element 
#9, Quality Information Feedback , section a. Quality levels.... 

A typical approach might be to: 1) review incoming, in-process, and 
finished product inspection data, 2) ask how, and if, this data is 
summarized and forwarded to the Quality control office, 3) ask the 
Quality control Manager to show you the file of data and and how, 
and if, the data is reported to other appropriate operation func- 
tions, 4) review the distribution and contents of the report sum- 
mary, 5) ask yourself: Are the right people on distribution? Is 
the summary clear? Is the frequency of reporting appropriate? Is 
there evidence of responses from the other functions?, 6) contact 
the recipients of the reports. Do they understand the reports? Do 
they think they need the reports? Is there evidence that they take 
action based on the reports? How do they see their responsibility 
to quality?, 7) review these questions, and answers, with the 
Quality control Manager and see if there is a mutual understanding 
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between Quality and the report recipients. You and the Quality 
Control Manager should jointly assess the effectiveness of the feed- 
back loop and the resultant action. If, however, the Quality Control 
Manager doesn't grasp the concept of this element, the recommended 
improvement program would entail upgrading management skills as well 
as instituting an effective system of information feedback. 

This is, of course, a simple example but it does highlight the 
need to determine if: 1) an adequate feedback system exists, 2) 
Quality Control and the recipients of the data have common objec- 
tives and understandings, 3) the system triggers the intended use- 
ful action. 

H. Summarize With Quality Control Manager 

Upon completing the audit, review your findings with the Quality 
Control Manager, with two objectives in mind: 

1) To ensure that your facts are correct. You may disagree 
on the conclusions; however, there must be no disagreement 
on the facts, otherwise the integrity of your report will 
be under suspicion. 

2) To give the Quality Control Manager an opportunity to 
reflect on the results so as not to be confronted by his 
boss without the facts. 

The success of future audits in that same facility will depend 
on how you conduct yourself in the first one. 

I . Summarize With Senior Management Within The Operation 

This is done for similar reasons as immediately above; but 
may be in different terms, depending upon the background of this 
level of management. 

J. Report 

Issue a written comprehensive report covering no more than what 
was discussed prior to the end of the audit; but including outstand- 
ing strengths within the organization, areas requiring improvement, 
and recommendations for those improvements. 

K. Follow-Up 

Assuming that the auditor will maintain contact with an opera- 
tion after the audit, the operation should issue a program for 
corrective action and subsequent progress reports. The auditor 
should use the original audit and appropriate progress reports as a 
basis for a future follow-up audit to measure the overall improve- 
ment. This phase is the true test of the effectiveness of an 
auditing activity. 
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QUALITY ASSURANCE INFORMATION SYSTEM AT KCL 


V.G.Malkhare 

Manager, Quality Assurance 
Kirloskar Cummins Limited 
Kothrud, Pune - 411 029. 


To quote Dr.Juran, * Quality Assurance is an activity 
of providing to all concerned the evidence needed to 
establish confidence that the quality functions are being 
adequately performed * . 

The confidence is established through an objective 
evidence and this evidence is derived from the documentary 
records. In other words, the document becomes an essential 
base of Quality Assurance. Mere documents alone do not 
sufficiently establish the confidence that quality functions 
are being performed, but the information derived from the 
analysis of such documents will finally become a controlling 
factor to serve the purpose of assurance to the consumer. 

At Kirloskar Cummins we have tried to base our Quality 
Assurance information system on this philosophy. Before I 
give information on the Quality Assurance information system 
at Kirloskar Cummins, let me explain briefly what is Kirloskar 
Cummins . 

The Kirloskar Cummins complex is situated at Pune, loo 
miles south of Bombay in India, employing approx. 1750 personnel 
and engaged in the manufacture of Cummins diesel engines of 
various models ranging from 60 to 800 h.p. 

Kirloskars are pioneers in the manufacture of Diesel 
Engines in India, and have been practising advanced quality 
control techniques in their organisations for quite a long 
period of time. The quality control movement in the engineering 
industry of our country owes its success to a very large 
extent to Kirloskars' foresight in adopting these techniques, 
successfully. 

The Quality Assurance information system provides three 
tier analytical reports every month. The basic purpose is 
to educate and involve personnel at all levels right from 
the operator to the top management in various quality activities 
at the plant. Consideration is also given to provide information 
which could be easily digested. 

We have processed the information available through 
Quality Systems, on EDP . When the first few Quality reports 
were released to Line Managers, one look at the heaps of 
computer printouts were a tough proposition for them to digest. 

It added to their confusion and all reports were neatly placed 
by them on the racks without any corrective action. It was 
realised that there is a communication gap and which required 
to be filled up. Further it was found out that sorting of 
information into 'vital few' and 'trivial many' was essential 
to fill-up the communication gap. This has to be done with 
prior knowledge of product and process and with understanding 
of what could be ' vital few'and which required correction. 

This was done manually. Manual analysis of data as a system 
was possible only because of its homogenous nature and 
comparatively smaller volume of production. 
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Initially the need for and the type of documentation 
required at various stages of quality control activities 
have been established. 

A reorganisation in the working set up of our Quality 
Control Division was required to ensure authentic and first 
hand information. 

The reorganised functions of Quality Control are explained 
briefly in Figure 1. 



There are some new responsibilities added to Quality 
Control which as a convention were not with them. This was 
done to widen the horizons of Quality approach. For example, 
Warranty administration and customer Liason which is a new 
addition. Certification of product by external certification 
agencies like Lloyds Register/American Bureau or Factory Mutual/ 
Under Writers Lab etc. was an additional responsibility. These 
new horizones were required for Quality personnel to enlarge 
their out look of Quality problems and get first hand knowledge 
of customers point of view about the product. This has also 
helped in effectively forming a Quality Assurance information 
system and implementing corrective action. 
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The new working set up facilitated compilation of data 
from documents and an analysis of first hand information 
proved useful in initiating corrective action. 

A primary report which is for the information at the operator 
level is a simple bar chart which is released every month to 
be exhibited on various production lines. This bar chart 
indicates the cost of controllable manufacturing scrap. It is 
impressed on the operator that had he avoided occurance of the 
scrap, the Company would have gained the money Figure 2 is 
a specimen of this reporting method. 



Figure 2 


This method of reporting has a direct impact on operators 
and lead to competition between Sections to keep the controllable 
scrap cost low. 

The second report of the three tier reporting system is for the 
use of the Line Manager or Foreman level. This is a middle 
management level and the corrective action implementation is 
always at this level. This report gives all the vital information 
of quality deficiencies in components manufactured in their section 
The Line Manager gets information on the rejection quantities 
of his section during a period, the reasons for rejection, the 
cost of rejection etc. He is further informed of the component 
performance, on the assembly and testing stage, which is machined 
and processed in his section. Through this report he also knows 
the field performance of the components manufactured in his 
section. Discrepancies observed during the product quality audit 
are also given to him for the corrective action implementation. 




In short a periodic quality information is available to him right 
from the manufacturing stage to the field performance. Figure 3 
represents a specimen of this report. 
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Line Managers and shop inspection supervisors hold meetings 
regularly to study the report and corrective actions are planned. 

Higher Management is reported with an overall view of the 
Quality activities in the plant and product performance in the 
field. 

The third tier information to higher management contains 
Quality Indices, quality trends, outgoing quality levels, as 
derived from audit results, quality costs of internal and 
external failures, significant quality problems which require 
higher management attention and corrective actions implemented 
as a result of weekly meetings. 

Separate reports are prepared every month for the procurement 
department which is provided with information on vendor 
performance. Vendor performance reports include the cost of 
rejection of supplies and vendor quality rating during that 
period and quality trends in the supplies. 

Apart from these, analytical reports on product reliability 
tests, quality systems audit and product field performance are 
released periodically. The idea is to provide accurate and 
exhaustive information to concerned departments for effective 
corrective action and avoid field surprises. 

Figure 4 indicates the source of information and the quality 
assurance information system at Kirloskar Cummins. 
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MINI-MIZING CALIBRATION PROBLEMS 


Cephas B. Rogers, Principal Engineer 
Polaroid Corp. 

Waltham, Mass. 02154 


OVERVIEW 

One view of a calibration system is to see it as an expense to be 
kept as low as possible. A second view sees it as a technician who 
shows up periodically to tinker with the measuring devices only to dis- 
appear minutes before the device stops working completely. Unifortu- 
nately both views are often brought about by the failure of the calibra- 
tion department to adopt a systems approach to its position in the 
overall Quality Information System. 

The following material describes a simple but effective mini- 
computer based calibration system which focuses on the following goals: 

- Providing top management with information on the dollar value 
of its instrumentation investment and the overall "health" of 
those instruments. 

- Providing operating departments with essential information on 
individual instruments. 

- Providing calibration personnel with an up to date data base for 
a) maintaining control, and b) effecting change towards improved 
levels of performance. 

THE SYSTEMS APPROACH 

A popular joke has an aircraft pilot announcing to his passengers 
as bad news that they are totally lost and running low on fuel and as 
good news that they are making excellent time. It is all too tempting 
to approach calibration control in the same manner. The enormous in- 
crease in both the quantity and complexity of instrumentation in the 
past decade has left many calibration departments barely able to keep 
up with day to day demands. In such an atmosphere it is not surprising 
to find that implementation of even modest systems improvements can 
result in significant dollar savings. 

As shown in figure 1, proper control requires a) knowledge of what 
is required, b) knowledge of current performance, and c) the means for 
doing what is required. 


MEANS 


KNOWLEDGE 


PERFORMANCE 


MEANS 


Figure 1. Basic Control Loop 
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TOP MANAGEMENT FOCUS 


In performing its function of optimizing profitability, top manage- 
ment operates best when it has facts on which to base its decisions. 

A successful calibration system recognizes and responds to this need. 

The data base in the MINI-CAL System contains the actual replacement 
cost for each instrument. When totaled, this figure often gives top 
management a clearer picture of the size of the instrumentation task 
than any other figure. Quantity, however, is also reflected by figures 
which show both the total number of instruments and the total number of 
measurement channels. The importance of the latter figure becomes 
clear when one realizes that some measuring instruments contain over 80 
separate measurement channels (each requiring calibration) . 

Thus by providing meaningful figures on cost and quantity, the sy- 
stem is able to relate to top managements desire for a dollarized, 
quantified measure of the task being undertaken. 

CUSTOMER FOCUS 

A user of instruments is much more likely to remember the one time 
an instrument failed than the hundreds of times it didn’t. Realizing 
that this is basic to human nature, the MINI-CAL System was developed to 
present meaningful reliability and performance figures for each depart- 
ment serviced. Measures of accuracy and precision (to be discussed 
later) as well as the number of both scheduled and non- s cheduled service 
calls are kept for each instrument. These figures are incorporated into 
monthly reports which are reviewed with the user departments thus engen- 
dering a close and mutually supportive relationship between the calibra- 
tion group and the departments being serviced. 

INTERNAL FOCUS 

The third area focused upon by the MINI-CAL System is that of the 
calibration department itself. Recognizing that clear cut lines of 
responsibility are essential for task management, each instrument is 
assigned to a specific engineer- technician team within the department. 
With this information as part of the data base, each team receives 
routine reports showing the status of those instruments for which they 
are responsible. Monthly reports are generated by the computer showing 
instruments due for service so that an efficient servicing schedule can 
be planned. In addition, a monthly exception report is generated show- 
ing instruments not calibrated on schedule as well as instruments having 
substandard reliability, accuracy, or precision. By highlighting those 
instruments with performance problems, the system provides a feedback 
loop for engineering corrective action as x^ell as tightened calibration 
schedules , 

The calibration manager, thus prepared with cost and quantity fig- 
ures for top management, with performance figures for departments ser- 
viced, and operating results for substandard instruments is able to 
maximize overall calibration effectiveness. 

SYSTEM HARDWARE 

A Hewlett-Packard model 9830-A Mini Computer with an INFOTEK Corp. 
16K Memory Expansion Supplement forms the hardware base for the system. 
In addition, a Hewlett Packard model 9867B Mass Memory and a model 9869A 
card reader are used for data storage and input respectively. Hardware 
costs were between $30K and $40K. (Note: While it has not been tried, 
the author is confident that most if not all of the features of MINI-CAL 
could be achieved using floppy disc technology for a total cost of under 
$ 2 OK . ) 

Time required to input all data and run all reports is approximate- 
ly 4 hours per week, thus the prorated har dwar e cost for the entire 
system is $500/yr. as shown below: 



40,000 Purchase Price 
+10,000 10 Yr Maintenance & Overhead 
$50,000 

x 0.10 Assuming 10 Yr Life 
5,000/Yr 

x 0.10 Fraction of 40 Hrs/Wk 

$500/Yr = Annual Hardware Cost for MINI-CAL 

DATA BASE 

The MINI-CAL data base consists of the following information ele- 
ments stored on the mass-memory disc for each separate instrument. 


1) 

Calibration ID// 

14) 

Procedure Availablity 

2) 

Description 

15) 

Location - Bldg. /Floor 

3) 

Asset // 

16) 

Dept 

. No . Used By 

4) 

Serial // 

17) 

User 

(Title or Initials) 

5) 

Drawing // 

18) 

Last 

File Update (Date) 

6) 

Type (Mech, Elec, Etc.) 

19) 

Last 

3 Repair Symptoms 

7) 

Number of Meas. Channels 

20) 

Last 

3 Repair Causes 

8) 

Replacement Cost 

21) 

Last 

3 Accuracy Reports 

9) 

Dept. Purchased By 

22) 

Last 

3 Precision Reports 

10) 

Date Installed 

23) 

Last 

12 Mos* - Sched. Calls 

ID 

Cal Due Frequency 

24) 

Last 

12 Mos* - Sched. Hrs. 

12) 

Responsible Engineer 

25) 

Last 

12 Mos* - Emer. Calls 

13) 

Responsible Technician 

26) 

Last 

12 Mos* - Emer. Hrs. 


* By Month 


While most of these data elements are self-explanatory, two of 
these deserve additional comment. 

Accuracy = Difference between Lab Standard and Local Reference 
Standard (Worst Channel) 

Precision = Range of 5 repeat readings on same part (Worst Channel) 

Due to memory size and programming considerations, it was decided 
to input to the computer only the worst channel results. In addition, 
both figures were converted to a per cent figure by dividing by the 
appropriate product tolerance. Thus: 

Accuracy (%) = .001" error _ 

.020" total tolerance 

With this feature as part of the data input all system users from 
top management to servicing technicians are able to judge the accuracy 
and precision "health" of the instruments. This in turn permits factual 
discussion concerning company goals for measurement error. Expressing 
accuracy and precision as a % of product tolerance also leads simple 
rules for establishing when an instrument is no longer fit for use, 
and must be repaired. 

Note: Programming and practical factors limited storage of accuracy and 
precision to six digits. This resulted in rounding all errors of 
100% or greater to 99%. Storage was thus limited to the last 
three checks. 

Example - Accuracy @ 159905 means: 

15% error at beginning of latest service 
>99% error at beginning of second latest service 
5% error at beginning of third latest service 
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FIGURE 2. INPUT CARD (FRONT) 



DWG.# ASSET# 

OTHER# . PRINC: MEOPDS 
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DEPT PURCH BY CAL. FREQ._ 

ENG# PROCEDURE: YES NO 

COMMENTS: ““ 


INSTRUMENT DESCRIPTION TECHNICIAN 


FIGURE 3. INPUT CARD (BACK) 











DATA INPUT 


After determining what data was needed by the computer, the ques- 
tion of HOW remained. How to input the data to the system so as to: 

a) Minimize Input Errors 

b) Minimize Input Effort 

In looking at the options of keyboard vs. card input, the advan- 
tage of card input became readily apparent when one considered the 
approximately 100:1 speed advantage of cards. Not having a keypunch 
convenient to the department determined that mark- sense cards would be 
used, although a strong desire for single document handling would have 
probably lead to the same decision. 

The card front (Figure 2) contains blocks for the following data: 

Col. 1 = Type of service call. E = Emergency; 

C = Calibration; P = Preventive Maintenance; 

L = Instrument Lost; I = Instrument Inactive; 

A = Instrument reActivated; M = Instrument Moved 
Col. 3-6 = Calibration Number 

Col. 7 = Optical subset designation within a cal. number. 

Col. 9-15 = Building/Floor Location of Instrument 

Col. 17-19 = Department number using Instrument 

Col. 21-22 = % Accuracy (worst case, before adjustment) 

Col. 23-24 = % Precision (worst case, before adjustment) 

Col. 26-27 = Symptom Code (if any) 

Col. 28-29 = Repair Code (if any) 

Col. 31-33 = Hours to complete service 
Col. 35-37 = Date service completed 

Col. 40 = Number of separate service calls covered by this 
card . 

The card back (Figure 3) is used for data recordings, as well as 
a source document for initial data on a new instrument. The columns 
PAR 1, 2 ... etc. serve for recording calibration data by paragraph # 
when referencing formal calibration procedures. The installation por- 
tion contains in addition to the obvious entries, the following coding 
for principles of operation: 

M = Mechanical P = Pneumatic 

E = Electrical D = Densitometric 

0 = Optical S = Sensitometric 

SYSTEM SOFTWARE 

The MINI-CAL software has been constructed to allow ease of data 
base manipulation and report generation. Programs written in BASIC 
provide the following functions: 

PUTIT: Interactive keyboard input of new instrument information from 

back of mark-sense card. 

MCARD: Batch feed of mark-sense cards through card reader to update 

data base. 

FIXIT: Interactive keyboard program allowing updating of any data base 

element . 

EXREP: Prints out monthly exception report for each building showing 

each instrument which: 

a) was not calibrated within the allocated frequency 
where W = Weekly, M = Monthly, A = Annual, etc. 

b) exceeded prescribed accuracy or precision limits. 

c) had more than 4 emergency service calls„ 



EXCEPTION REPORT - BUILDING Rl 


DEC 1977 PG 1 

EMER LAST 

TECH CAL// DESCRIPTION ACCURACY PRECISION CALLS CAL FR 


14 49 SPEED OF PULL 153020 * 0 771116 M 


19 1451 RADIOMETER * 


010624 

0 

* 


W 


- SUMMARY - 







COMPLETED 

MISSED 

AC/PR 

EMERGENCY 

CALLS 


INST 

CHECKS HOURS 

IN/CK 

>20% 0 

1 

2 

3 

4+ 

EMERGENCY 5 

5 6.4 

— 

152 

5 

0 

0 

0 

SCHEDULED 75 

130 88.0 

3/3 

3 

— 

— 

__ 

- 

TOTAL INST SERVICED = 77 

TOTAL 

INST = 252 





* = Within 

specified limits this 

month 






Figure 4. EXREP Report 


FULREP : Prints out monthly report by building showing calibration 

status of all instruments at that location. 



ALL INSTRUMENTS 

BUILDING Rl 

MONTH OF 

DEC 1977 


TEC 

CAL// 

DESCRIP 

ACC 

SCHED 

PREC // HRS 

EMER 
// HRS 

LAST 

CAL 

FR 

LOC 

14 

85 

AIR GAGE 

101010 

201515 1 0.3 

0 


771207 

M 

Rl - 2 

19 

155 

CAP GAGE 

100515 

152005 0 0 

1 

0.5 

771031 

M 

Rl- 2 



TOTALS 


135 94 

11 

10 





Figure 5. FULREP Report 


ALPHA: Prints out an alphabetic listing of all instruments in the 

sys tem. 



DESCRIP 

DWG// 




COST 

ENG 
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PCD 

FR 

Hi 

AIR GAGE 

A283521 




3 , 0K 

2 

21 

N 

M 
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XRAY CAP 
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SUP 

1 . IK 

7 

14 

Y 

A 


TOTAL INSTRUMENTS = 1127 

TOTAL MEASUREMENT CHANNELS = 2072 

TOTAL INSTRUMENT COST = $2,421,200 


Figure 6 


ALPHA Report 








CALCHK: Prints out activity report covering last 12 months. 


#26 POD 

TESTER 

FREQ = M 
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EMER 
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1 

1 

1 

1 

1 

1 

1 
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EMER FR 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

0 

EMER HR 

0.0 

0.0 

0.0 

1.0 

0.0 

0.0 

1.5 

0.0 

0.0 

0 

0.0 

0.0 


Figure 7, CALCHK Reports 


TECHS: Prints out report of all instruments assigned to a given 

technician. Instruments due are designated by *; those 
missed the previous month are designated by **. Each tech- 
nician is given a printout at the first and third weeks of 
every month. 
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Figure 8. TECHS Report 


NUMREP : Numerical listing of all instruments in the system (similar 

in format to ALPHA) . 

DEPREP: Lists all instruments purchased by and/or used by a given 

department (similar in format to FULREP), 

CONCLUSION 

This paper has described one company’s success in implementing 
a mini-computer based calibration system. While return on investment 
figures for prevention activities are difficult to substantiate those 
managers who are familiar with MINI-CAL agree that it has been one of 
our most successful systems engineering ventures. And finally those 
who rely on the data from the instruments serviced by MINI-CAL sleep 
just a little bit better each night. 
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CONSUMER PROTECTION ORIENTED SAMPLING PLANS 


JOSH B. TYE , QUALITY MANAGER 
DAISY, SUBSIDIARY OF WALTER KTDDE & COMPANY, INC. 
ROGERS, ARKANSAS 


INTRODUCTION 


The only quality level acceptable for defects which cause injuries is zero. It 
has long been recognized that in most industries this is not attainable. The producer 
has the responsibility for maintaining the quality of his product, and through 
knowledge of his product and of the relationship of the defect to the injury, he can 
pick the most realistic sampling plan to give the consumer the level of protection he 
needs . 

This document establishes sampling tables designed such that the emphasis is on 
the prevention of defects from reaching the consumer. Contrary to the traditional 
Quality Assurance approach in the design of sampling plans, economics has been 
sacrificed in favor of accuracy. For ease of application, only single sampling plans 
are used. It may be used in conjunction with 100% inspection, or under proper 
circumstances, as an alternative. These plans are to be used as part of a Quality 
Assurance program. Total reliance on them does not guarantee that no defective 
product will reach the consumer. 

Recognizing that the most published sampling plans are oriented around the top 
end of the 0C curve in order to minimize the producer's risk, it is obvious that the 
consumer's risk is allowed to vary a great deal. In the application of sampling plans 
involving defects that could cause injuries, the control needs to be oriented around 
the lower end of the curve. Because a number of variables are involved, several 
levels of protection are required. The most important variables to consider x*hen 
choosing a sampling plan are: 1) The probability of injury given the defect exists; 
and 2) The severity of the injury. 

Because the most popular and most widely accepted current publication of sampling 
plans is the MIL-STD-105D, the layout and format of the MIL-STD-105D is used in this 
document. Terminology and procedure were copied from MIL-STD-105D whenever possible. 


EXPRESSION OF LEVEL OF QUALITY 

AOQ and AOQL - AVERAGE OUTGOING QUALITY and AVERAGE OUTGOING QUALITY LIMIT. 

The AOQ is the average quality of outgoing product including all accepted lots or 
batches plus all rejected lots or batches after the rejected lots or batches have been 
effectively 100% repaired or inspected and all defectives replaced by nondefectives. 
The AOQL is the maximum of the AOQ's for all possible incoming qualities for a given 
acceptance sampling plan. To estimate the total defects reaching the consumer during 
any given time period, multiply the AOQ times the total product shipped. If the AOQ 
is not known, use the AOQL. 

TQL - TOLERANCE QUALITY LEVEL. 

The TQL is the maximum permissible fraction defective of any single lot. 

P (A) -- PROBABILITY OF ACCEPTANCE. 

A lot having a process average (Fraction Defective) equal to the TQL selected 
will have a probability of P(A) of 0.05 of being accepted. This indicates that 95% 
of lots that are this percent defective will be rejected. 
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SELECTION OF SAMPLE 


UNITS OF PRODUCT 

The unit of product is the item inspected for the purpose of determining its 
classification as defective or nondefective or to count the number of defects. It 
may be a single article, a pair, a set, a component, or an end product. 

LOT OR BATCH 

The term lot or batch shall mean a collection of units of product from which the 
sample is to be draxm and inspected to determine conformance with the acceptability 
criteria and may differ from a collection of units designated as a lot or batch for 
other purposes. 

FORMATION OF LOTS OR BATCHES 

The product shall be assembled into IDENTIFIABLE lots, sub-lots, batches, or in 
a manner prescribed by usual Quality Control practices. Each lot or batch shall, as 
far as practicable, consist of units of product of a single type, grade, class, size 
and composition manufactured under essentially the same conditions and at essentially 
the same time, and must be capable of being isolated. 

LOT OR BATCH SIZE 

The lot or batch size is the number of units of product in a lot or batch. 

SAMPLE SIZE 

The sample size is the number of units of product selected to be inspected. For 
any given lot size, the corresponding sample size may not be smaller than indicated 
in Table I. 

REPRESENTATIVE SAMPLING 

Samples must be representative. They may be random or stratified or a 
combination of both. 


ACCEPTANCE AND REJECTION 
ACCEPTABILITY OF LOTS OR BATCHES 

Acceptability of a lot or batch \<rill be determined by the use of a sampling plan 
or plans associated with the designated AOQL and TQL. 

RESUBMITTED LOTS OR BATCHES 

Lots or batches found unacceptable shall be resubmitted for reinspection only 
after all units have been screened and all defective units removed and/or repaired. 
All screened lots must be resubmitted. Screened and reworked lots should be 
subjected to the same sampling plan as new lots. 


INSTRUCTIONS 


1. Determine the AOQL. 

If the minimum acceptable number of defects allowed to reach the consumer during 
any given period of time can be determined, divide this number by the total 
production for that period of time. 

2. Turn to Table I. — Using lot size and AOQL, determine the Sample Code Letter 
and TQL. 

In the body of the table, the information is in the form of a letter, a decimal 
point, then numbers. 

Example: For a lot size of between 2,001 and 7,000, when an AOQL 

of 0.0001 (1 in 10,000) is desired, the sample code letter 
and TQL are R.001. The letter is the Sample Code Letter, 
and the number is the TQL. 



TABLE I - SAMPLE SIZE CODES 



3. Turn to Table II. — Using the Sample Code Letter and TQL, determine the sample 
size and Acceptance and Rejection numbers. 

Example: R.001 

Read down the extreme left hand column (headed Code Letter) 
until the letter R is found. In the next column to the right, 
the sample size of 3070 is found. Next read down the column 
headed (TQL =) 0.001 until it crosses the row containing R and 
3070. This intersection (0,1) contains the Acceptance and 
Rejection numbers. More simply, R.001 indicates a sample of 
3070 is to be inspected, and the lot accepted if no defective 
units are found or rejected if one or more defective units 
are found. 

AOQL's are displayed on the right side of the table. A different combination of 
Code Letter and TQL may be selected SO LONG AS THE SAMPLE SIZE IS NOT REDUCED. 





TABLE II - CHART OF TOLERANCE QUALITY AND AVERAGE OUTGOING QUALITY 



USE FIRST SAMPLING PLAN BELOW ARROW 


USE FIRST SAMPLING PLAN ABOVE ARROW 


AC ACCEPTANCE NUMBER 
RE REJECTION NUMBER 


WHEN THE SAMPLE SIZE EXCEEDS THE LOT SIZE, 100% INSPECTION IS REQUIRED 


SWITCHING PROCEDURES 

The charts on the following pages are OC curves and AOQ curves that represent 
all of the families shown in Table II, grouped by TQL. When the producer has used 
the prescribed sampling plan for a sufficient period of time to be confident in his 
estimate of the process average, he may refer to the charts and choose a smaller 
sample size having a comparable consumer’s risk and a producer’s risk that he feels 
he can live with. Switching should be accomplished only after a sufficient number 
of lots (at least ten) have been accepted consecutively using the current plan. Any 
time a lot is rejected, the original plan should be resumed and the entire procedure 
started over. The decisions surrounding switching, like the decisions regarding the 
choice of AOQL and TQL, are the responsibility of the producer and are based on his 
knowledge of the product and the process. 


LCS : 222:70:434 
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CONSENSUS STANDARDS: A PROFESSIONAL VIEWPOINT 

By Russell J. Eilers, M.D. 

Medical Director, Bio-Science Enterprises 
Van Nuys, California 91405 


The clinical laboratory industry is only one segment of the most 
rapidly growing industry in the United States - the health care industry! 
Expenditures for health in 1976 were 8.6% of the Gross National Product 
(GNP) . Whereas the GNP increased 18% during the last two years, expendi- 
tures for health increased 31%. Public funds accounted for 40% of this 
expenditure and increased 15.6% in 1976. This is changing the focus of 
the limelight interest of government and Congress from ways and means to 
protect the public health welfare to their economic welfare to control 
this growing public fund cost for health. This is essential if the 
National Health Plan objective is to be accomplished as a political 
promise of the current administration by 1980. 

The clinical laboratory accounts for only 7% of the health expendi- 
ture in contrast to 40% for hospital care, which is increasing more 
rapidly than the 20% for physician care. As an ancillary service to 
both, the clinical laboratory industry is a likely candidate for control. 
Witness to this the impact of the 1966 Medicare Act, Clinical Laboratory 
Improvement Act of 1967 and the Medical Device Act of 1976 on the 
clinical laboratory industry. All of these laws and subsequent regula- 
tions have involved and will further involve the process of 
standardization for the field. Many of the available voluntary standards 
(criteria or specifications) were adopted by government and made 
regulatory standards. Some were associated with other government written 
criteria to become the value standards required by Medicare and CLIA-67. 


The individual pathologist, as the physician in charge of the 
medical laboratory, and organized pathology have long been involved in 
the establishment of "standards 0 . These have evolved into a six phase 
management program of total quality control for the medical laboratory: 


I. DESIGN CONTROL : Facility, Staffing, and Assays for Mix of Health 
Care Problems. 


II. RAW MATERIAL CONTROL : Standards, Controls, Reagents, Instruments, 
Glassware, Specimens, Personnel. 

III. PROCESS CONTROL : Internal Program for Calibration, Process Control, 
and Preventive Maintenance; External Monitoring by Proficiency 
Testing and Inter-Laboratory Comparison Program of Accuracy and 
Precision. 


IV. OUTPUT CONTROL : Accurate Results in a Timely Manner and a 
Medically Significant Format. 

V. RELIABILITY CONTROL : Assay Utilization Correlates with Health 
Care Needs. 

VI. VERIFICATION CONTROL : Inspection and Accreditation, Workload 
Analysis; Cost Analysis. 


Initially, with few parameters of testing procedures to select from, 
the physician found it relatively easy to select the proper test and 
method for a particular problem. The pathologist's time was devoted 
to searching for new parameters to investigate disease that could be 
easily done by laboratory staff and developing quality control 
mechanisms for Phase II and Phase III. Specifications were developed 
for chemical standards (bilirubin, cholesterol, cyanmethemoglobin ) . 

A system for process control at the bench was developed. Organizations 
1978 American Society for Quality Control 
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became interested in survey of the "state of the art" of laboratory 
testing and how to improve it by further education and training of the 
pathologist and his technical staff. 

Organized pathology devoted much attention to criteria for the 
educational requirements, training and certification of laboratory aids, 
technicians, and technologists. The criteria provided for education 
and training of generalists in medical laboratory technics. The 
specialty interest of technologists and their expertise developed with 
years of experience on the job and eventually promotion to supervisors 
in a particular discipline area. Relying on these experienced super- 
visors to bring problems for solution to his attention, the pathologist 
concerned himself mostly with those procedures that demanded his active 
participation, such as the interpretation of peripheral blood and bone 
marrow smears, problem body fluid sediments, tissue sections and 
cytologies. This daily routine allowed the pathologist to easily 
monitor many of the qualitative and quantitative tests associated with 
these procedures. The proper format for presentation of results to his 
physician colleagues for reliable interpretation of results was 
relatively simple due to the limited number of parameters available. 

With the ever increasing variety of laboratory parameters and the 
volume desired by clinicians to resolve their patient care problems, the 
quality control of the flow of workload through the medical laboratory 
became more complicated. Yes, this necessitated the six phase manage- 
ment program outlined above. The new parameters required new 
instruments utilizing diverse physical principles in their measurement 
technics and marked diversification of reagents and growing dependency 
on industry to make them for the laboratory, frequently in the form of 
kits for a complete assay. Yes, overall quality control of the process 
was now being shared with the outside in-vitro diagnostic industry. 
Hence, the growing need for the Medical Device Act. The more 
complicated methods required additional personnel with Masters or Ph.D. 
degrees in their specialty areas, demanding new personnel requirements 
and working relationships and creating gray areas of supervision 
responsibility. Automation allowed the handling of increased volume of 
routine laboratory tests and the demand for some of the more 
sophisticated tests. With this mass production of results via automa- 
tion, the necessity for computerization to aid in calculations and 
simply to just handle the volume of data being generated raised new 
questions of format of presentation of results to the physician for 
reliable interpretation of results to assure quality patient care. 

Should the format be decided by the ease of programming and economic 
cost or by the logical physiological groupings to aid interpretation in 
defining disease or monitoring the effects of therapy? Likewise, 
should data input require a new cadre of clerks and key punch operators 
to handle this via alpha-numeric terminals only? Or could instrument 
interfaces or terminals compatible with the bench technologists work be 
designed to allow their direct communication with the computer, thus 
maintaining their control of result reporting, but improving their 
productivity? 

The automated profile testing to elicit hidden or unknown disease 
has evolved an attitude that the high volume production of accurate and 
precise numbers in Phase III is the practice of laboratory medicine. 

This has pinpointed the need for developing criteria or "standards" for 
the selection of the proper analyte and test method for a particular 
patient problem. Do you do 5, 10, 12, or 27 tests because they can be 
automated together for the sick and healthy? Yes, medical usefulness 
criteria for proper utilization of laboratory tests. The principles 
of mass production of industry can be applied to make possible the 
availability of accurate and precise numbers m high volume at a reduced 
cost. Of course, this is based on the assumption that the volume for 
these tests exists in a particular hospital or the test can be sent out 
to a regional laboratory to create the volume. The inspection and 
accreditation agencies in Phase VI now need to define criteria, not 
only for quality control of the result, but the necessity for doing a 
particular procedure on-site within a specified turnaround time to meet 
the physician's need for reliability in Phase V. Yes, individual 



pathologists and their organizations will be interested in the further 
establishment of standards relating to all six phases of total quality 
control . 

What will "standard” mean in each of these six phases and to each 
of the diverse people involved in or utilizing the medical laboratory. 
Even Webster's dictionary provides a diverse set of meanings for this 
term standard; for example, rallying point, personal emblem, criteria 
by authority, measure of quantity or quality by authority or legally 
fixed weight. The evolutionary history of the clinical laboratory 
field, as other industries, has led to diverse meanings for the term 
standard within the field. 

TO INDUSTRY : Standardization provides a mechanism for dollar savings 
through mass production by production of uniform goods and reduction of 
time and materials through standard designs, equipment, procedures and 
testings. Opportunity for areas of profit if volume sales can be 
assured. 

TO PURCHASING AGENTS ; Standardization offers increased efficiency by 
freeing him from the need of preparing individual sets of specifica- 
tions and descriptions for each purchase. 

TO NATIONAL PROFESSIONAL TECHNICAL AND TRADE ASSOCIATIONS ; Voluntary 
standardization allows them to bring their consumers' interests into 
consonance with current technology, engineering and manufacturing 
practices and laboratory and medical practices. 

TO FEDERAL GOVERNMENT : As a vast business enterprise, standardization 
to them, is a mechanism to regulate government purchasing of commodi- 
ties and services and fulfilling value objectives and standards for 
society. 

TO PHYSICIAN CONSUMERS : Standardization means the availability of 
reliable laboratory results that are consistent day to day and from one 
institution to another. 

TO THE NATION'S ECONOMY : Standardization pays off in greater profits, 
higher quality goods' ami customer good will. 

TO THE WORLD : Standardization holds the key to the exchange of goods 
and services in international trade. 

TO THE COMMON MAN ; Standardization supports his belief in the nation's 
system of free enterprise to give him the best quality at the lowest 
reasonable price and, therefore, a higher standard of living. 

To allow the development of voluntary standards for the medical 
laboratory in face of these diverse meanings for standards, the 
College of American Pathologists put forth the call in 1966 for the 
formation of the National Committee for Clinical Laboratory Standards. 
Working upon previous experiences of the CAP Standards Committee in the 
field and the success of consensus making standard agencies in other 
industries as follows: 

Search within the science and practice should define the need for 
and probable basis for a standard to improve the quality of laboratory 
practice and, therefore, patient care. The basis for the standard 
would have to be tested by all interested parties within and outside 
the field - government, industry and professions. Analysis of the 
results and findings should lead to the narration of a set of 
specifications for a proposed standard, which must be announced to the 
clinical laboratory field. Time must be allowed to discuss the set of 
specifications through open forum and written communications among all 
parties involved. Justified criticism and suggestions could thus be 
obtained for analysis to lead to narration of a revised set of 
specifications, which should be annotated to define the standard. 
Verification of consensus of interested groups should document the 
specifications as a voluntary consensus standard for the medical 



laboratory. Then comes the hard task of implementing that standard in 
the practice of laboratory medicine. 

The implementation of the voluntary standard in the private sector 
is only assured when all interested parties in the consensus educate 
their personnel and apply the standard in their segment of the clinical 
laboratory industry. The implementation of these voluntary consensus 
standards should be so freely done and well done within the private 
sector that implementation by law and regulation is only a by-product 
of the voluntary effort when policing action is necessary. 

Considering the great variety of standards needed in the clinical 
laboratory industry, have the interested parties agreed on a co- 
ordinating focus for developing consensus standards or should there 
even be consensus standards for certain areas (personnel, facility 
design, modes of practice) ? Agreement on some consensus standards 
has been reached via NCCLS, others by regulatory action in the U.S. 
or pressure from international professional bodies. Some must be 
accepted as consensus standards as established previously for other 
industries via ASTM or ANSI. There is a need for agreement for NCCLS 
as a focus for defined types of necessary standards for the clinical 
laboratory industry. In turn, a liason must be established with ASTM 
for those standards overlapping various industries and with ANSI as the 
acceptance focus for U.S. consensus standards. From ANSI, U.S. 
standards could then proceed to various international bodies depending 
upon their types and international interests to be obtained. Yes, 
whither go biomedical consensus standards - to I.S.O., W.H.O. , or 
international professional groups. 
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VOLUNTARY CONSENSUS STANDARDS IN A REGULATORY CONTEXT 


Michael J. Miller, Executive Director 
Association for the Advancement of Medical Instrumentation 
Arlington, Virginia 


The purpose of this paper is to point out some of the differences 
between voluntary consensus and regulatory medical device standards and 
how these differences should be considered during the voluntary standards 
development process. 

The Medical Device Amendments of 1976 (PL 94-295) draws a clear 
distinction between voluntary consensus standards and mandatory standards 
adopted by the Food and Drug Administration. The distinction between 
these two types of standards will affect the process by which standards 
are developed. 

Because of the significant differences between voluntary consensus 
standards and mandatory standards, many voluntary consensus standards 
will not be acceptable for regulatory purposes or will have to be 
significantly revised to be used as regulatory standards. 

Consequently, voluntary consensus standards organizations should 
clearly understand the differences in purpose and effect of voluntary 
and regulatory standards. 

Many voluntary consensus standards express, in general terms, 
desirable technical concepts which establish a frame of reference for 
safety and effectiveness. While establishing this general frame of 
reference for voluntary purposes may be desirable, attempts to enforce 
strict compliance with such standards under regulation could impose 
severe or impossible conditions of compliance on both users and 
manufacturers. For example, some voluntary standards may include 
requirements not within the present state of the art or requirements 
so expensive that implementation would be impractical. 

Voluntary consensus standards, by definition, do not require 
absolute conformance on the part of the manufacturer or user. Voluntary 
consensus standards that are later put through proper FDA procedures and 
become regulatory standards require strict compliance by the manufacturer 
or user in order to avoid criminal and civil liabilities. 

Section 514 of the medical device law authorizes the FDA to prom- 
ulgate regulatory standards only on attributes of devices that (a) should 
be subject to regulatory standards and (b) will help assure the safety 
and effectiveness of devices. Unless there is an identified or potential 
safety or effectiveness problem with one or more device characteristics, 
Section 514 would not permit a regulatory standard to be set. 

Many voluntary consensus standards deal with attributes of devices 
that may not, in a regulatory sense, help assure the safety and effec- 
tiveness of devices. Consequently, their use for regulatory purposes, 
in terms of purpose and effect, would be inappropriate. 

The word "standard” should be considered in its broadest context 
to include guidelines, protocols, practices and procedures, specifica- 
tions, performance and design standards, test methods, and any other 
voluntary consensus documents that directly or indirectly could be us€?d 
to impose safety and effectiveness requirements on manufacturers and 
users . 


It is unclear at the present time exactly what use the FDA will make 
of voluntary standards. It is clear from a reading of Section 514 that 
a voluntary consensus standard could be used as the starting point toward 
©1978 American Society for Quality Control 
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the development of a regulatory standard. For this reason, voluntary 
standards organizations may wish to determine at an early stage whether 
standards development efforts are intended to result in documents that 
should or should not be used in a regulatory context. 

The purpose of this paper is to point out that since voluntary 
consensus standards may become the basis for regulatory standards, vol- 
untary consensus efforts should clearly document voluntary consensus 
standards with clear statements of rationale and scope so the documents 
intended only for voluntary purposes (where regulatory consideration 
of safety and effectiveness are not involved) will not be construed as 
potential regulatory documents. 

For example, purchase specification standards agreed on by manu- 
facturers and users should not generally be regarded as regulatory 
standards. To explain further, a specification standard reflects an 
understanding between a manufacturer and a user for certain products 
primarily for communication and convenience and not primarily for regu- 
lation of safety and effectiveness, except in a secondary sense. 

The purpose of this paper is not to discourage voluntary consensus 
efforts. The purpose is to begin to draw a clear line between voluntary 
consensus documents and potential regulatory documents and to provide 
guidance and assistance to the FDA as they consider voluntary consensus 
documents which may or may not be used for regulatory purposes. 

While the FDA must strictly adhere to the legislative authority 
outlined in the medical device law in reviewing, revising, and adopting 
voluntary consensus standards as regulatory standards, a clear under- 
standing of the differences between the two documents and the reflection 
of this difference in the developmental process, clear statements of 
scope and rationale, and good communications will help avoid the adop- 
tion of voluntary consensus standards that were not intended to be used 
as regulatory documents. 

I hope this paper makes it clear that responsible standards writing 
and approval can only be undertaken with a clear and full understanding 
of the Medical Device Amendments of 1976 (particularly Section 514) and 
its potential relationship to voluntary standards. There are many 
summaries of this law available and AAMI will send copies on request. 
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LABORATORY QC: DOCTOR, IS THAT RESULT ACCURATE? 
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INTRODUCTION 

St. Luke's is an acute care general hospital having over 500 beds and serving a 
metropolitan area. The clinical laboratories are staffed 24 hours a day seven days a 
week, by more than 250 people. In 1977, over one million determinations were performed. 
Approximately 50% of these tests were done in the Chemistry, Radioisotopes and Physio- 
logical Monitoring Laboratories. 

This paper describes the quality control programs designed for these three labora- 
tories. The programs in their computerized form have been in use since November 1972. 

Any QC program designed for these areas must have as primary goals accuracy and 
precision of test results with a short turn-around time. The program must also be able 
to verify that the same determination performed on another shift by a different tech- 
nologist using a different method and/or instrument will yield statistically similar 
results. The program in use documents control status by test, shift, instrument, 
method and date. No matter when the doctor orders a test on his patient, be it first 
shift on Monday or third shift on Saturday, he can be confident of the results. The 
answer he obtains reflects the condition of his patient, and thus is not a function of 
the abilities of the technologist performing the determination, the reagents, the 
instrument or the method used, and is what our program verifies. 

INDUCTIVE QUALITY CONTROL 

Industry uses various methods of QC based upon direct measurements and/or destruc- 
tive testing. Clinical QC cannot use these methods. In addition to being unable to 
test the constituent directly, are the added problems of small sample size, sample 
deterioration, special handling necessary for preservation and pressures to produce 
results quickly in life and death situations. 

The method we use is based upon inductive reasoning. Induction is an instance 

of reasoning from a part to a whole, from particulars to generals. ^ It has become 
common practice in well run clinical laboratories to incorporate various systems of 
controls, pools, replicates and/or split samples as part of their on-going quality 
assurance program. Each of these systems is based upon inductive reasoning. If the 
controls are running "in control," then the process is in control, therefore the 
results of the unknown specimen run in the same batch is also thought to be correct. 

However, there are times when the control sample is considered out of control when 
in fact the process is in control. Or the reverse, there are times when the controls 
appear to be "in" when in fact the process is out of control. In actuality we can be 
wrong in both instances. But how do we know this? 

There are also economic factors. Over control can become prohibitively expensive, 
and not enough control gives no assurance of precision or accuracy. How then, do we 
strike a balance that gives us assurance of the precision and accuracy of our results, 
has a short turn-around time and yet is economically feasible? 

AN OVERVIEW 

We feel that the system we have devised addresses all four areas of quality 
assurance: precision, accuracy, speed and economy. The program consists of four 
parts: 1. normal and abnormal serum controls in each run; 2. plots of the controls 

on modified Shewhart charts 3. analysis of variance (ANOVA) ^ and 4. tabulation 

of the mean, standard deviation and upper control limit of the range for each test by 
method. 

©1978 American Society for Quality Control 
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Use of normal and abnormal control sera verifies both areas of medical concern. 

The Shewhart range and average charts satisfy the questions of precision and accuracy, 
and shows when we have method trends, drift and/or change. The ANOVA shows day to day 
and within day variation. The mean, standard deviation and upper control limit of the 
range list are quick checks on overall process. 

Usually, two commercially prepared unassayed sera serve as control samples in each 
run. One control is in the normal range of the test and the other in the abnormal 
range. We contract for sufficient amounts of these pooled sera so one lot number is 
available to us for approximately 18 months. Two months prior to the end of a lot 

number we start assaying the new lot. This gives us time to obtain sufficient data for 

the calculations of means and standard deviation and the generation of the charts. 

INPUT DATA 

Data Card 

All data generated are logged on specially designed 11 by 13 3/4 inch sheets lined 
as 80 column data cards. See Figure 1. The following column headings are used for 
identification on each chart: columns 1-6 test name, column 7 control designation, 

column 8 number of plot points per day to be used in the ANOVA calculation, columns 9 - 

13 method, column 14 shift designation, columns 15 - 16 month, columns 17 - 18 are pre- 
printed with the date and columns 19 - 20 the year. The remaining 60 columns are used 
in groups of four for listing of control results. The words PLOT appear above columns 
21 - 24 and 33 - 36. These are the results that are averaged and recorded on the 
Shewhart charts. Figure 1 shows that CL (chloride) is the test, 2 is the control 
designation and four pieces of data per day are to be used in the ANOVA calculation. 

The method is manual and the date December 1977. 


The ANOVA program is written such that only those days with the designated amount 
of data are used in the calculation. Therefore column 8 is filled in after the data 
are recorded. This allows us to take advantage of this program feature. For example, 
looking at Figure 1, if we list 2 in column 8, 62 pieces of data would be used in the 
ANOVA, a 3 would put 93 pieces into the calculation, a 4 would put 120 into the calcu- 
lation, a 5 would put 135 and a 6, 114 pieces of data into the calculation. Therefore, 
the best number in this case would be 5. It can be seen that the data sheets serve the 
triple function of a log for the QC data, at-a-glance check of control status and as 
data cards for key punching. 

The computer program that has been developed to produce our system is a result of 

(4-7) 

the combined ideas of Riddick, Storey, Banker and Eichelberger with programming 

done by the Computer Center personnel at the University of Wisconsin. Milwaukee. 

Shewhart Charts 

We will describe the construction of the Shewhart range and average charts as this 
is a system routinely used in industry but rarely in the clinical laboratory. 

When a new lot of control sera is being checked it is run along with the old lot, 
twice a day for 25 days, for each parameter checked. If the old lot is in control the 
value for the new lot is accepted and logged on a data chart. The sum ( £ ) , average 
(X) , and range (R) of each subgroup (k) is then calculated. Plotted points on our 
average chart are always the average of two individual control readings. 

The following formulae are used in the calculation of limits for the average and 
range charts after the original data has been collected. 


LEGEND 


FACTORS FOR CONTROL LIMITS 


n = total number of observations 
£ = sum 

k = number of subgroups 
X = each individual observation 


2 


1.880 


3.268 


1.128 


1. Average (X) 

x = rx 

n 

2. Grand Average (X) 

X = Z X 
k 

3. Average Range (R) 

R = E. R 

k 

4. Upper Control Limit of the Range (UCL R ) 

UCL = D.R 
R 4 


5. Estimated Standard Deviation ( ^ ) 



6. Upper Control Limit of Average (UCL-) 
UCL- = X + A 2 R x 2/3 

7. Lower Control Limit of Average (LCL-) 
LCL- = X - A 2 R x 2/3 




QUALITY CONTROL CHAR 
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Figure 2 shows the Shewhart chart generated by the data in Figure 1. The chart 
heading gives the test name, control designation, method, shift, date, level and test 
units. The values for standard deviation (SD) , standard error (SE) , average, and 2 SE 
limits are preprinted on the average chart. The average range (R) and upper control 
limit of the range (UCL ) are preprinted on the range chart. Days for both the average 
and range are along the ^horizontal axis. Deviation from the mean and range respective- 
ly are on the vertical axis. All plotted points on the average chart are the average 
of columns 21-24 and 33 - 36 from the data card. 

Control limits for the average chart are SE units because averages are plotted. 

The upper and lower control limits of the average are set at +2SE rather than 3SE 
because of the dire consequences if the process is out of control. This is a conven- 
tion generally accepted in the clinical laboratory field. Plots falling in the +0 - 
1.5 SE area are considered ok, those in the + 1.5 - 2.0 SE area are in the watch zone, 
and those past the 2 SE limit line need action. The UCL R is at 3 SD. 

When the technologist checks an INDIVIDUAL control value it must be within +2 SD 
of idle mean to be acceptable. The AVERAGE of the two plotted points must be within 
+2 SE of the mean. 

OUTPUT DATA 

Each test for which data have been submitted receives the following printouts: 1. 
A plotted range and average chart; 2. A blank range and average chart with limits 
generated from the submitted data; 3. An ANOVA table; and 4. A table giving that 
month's results by test, method, mean, SD and UCL R . 


DATA FOR TEST CL 2 MACHINE MAN 1 DATE 12-77 


ANALYSIS OF VARIANCE TABLE 


SUM OF SQUARES DEGREES OF FREEDOM MEAN SQUARE F-RATIO SOURCE 


313.0000 

120 





TOTAL SUM OF SQUARES 


143.0083 

1 





CORRECTION DUE TO X OF 

98.9075 

169.9917 

119 





CORRECTED TOTAL SUM OF 

SQUARES 

56.2417 

29 



1.939 

1.568 

DATE 

6.0917 

3 



2.031 

1.642 

TEST 


107.6583 

87 



1.237 


RESIDUAL 


TEST MEAN 

FREQ 

TEST 

MEAN 

FREQ 

TEST MEAN 

FREQ TEST MEAN 

FREQ 


* A**************************************************************************************** 

TESTl 98.70 30 TEST 2 99.27 30 TEST 3 98.73 30 TEST4 98.93 30 


GRAND MEAN FREQ 

******************* 
98.9075 120 


STANDARD DEVIATION 
******************* 

1.112 


FIGURE 3 

Figure 3 shows the ANOVA table generated from the data. Test identification is on 
the top. Then the usual ANOVA table showing the sum of squares, degrees of freedom, 
mean square, F ratio and source is displayed. Below this is given the mean and fre- 
quency of each column followed by the grand mean and SD. 


TEST SUMMARIES 


TtST N 

AV£ 

MACHINE 

MEAN 

ST. DEV.’ 

RANGE ucl 

SAL.. 

2 

GAG 

23.0 

.206 

.221 

S GO T 

1 


14.1 

2.0 62 

4.638 

S GOT 

c. 

gma 

JLCLl.J 

4.2L4 

9.171 

SGOT 

f. 

AO A 

335.9 

5.143 

1 1 .912 

-GOT 

N 

ACA 

24.7 

.977 

3.268 

SGPT 

E 

AC A 

133.8 

1 .740 

0.000 

SGPT 

N 

ACA 

29.3 

.PRO 

0.000 

tril 

1 

ACA 

1.5 

.076 

0.000 

trtl 

\ 

GAS 

^ 9 5 

• 050 

. 042 

TBIL 

2 

ACA 

4.7 

.154 

0.000 

TRIL 


GAS 

4.7 

. 1 64 

.084 

7 IPC 

1 

ACA 

376.3 

5.023 

0.000 

7 P 

) 

ACA 

7.0 

. OR 9 

0.000 


FIGURE 4 


Figure 4 shows a portion of the test summary. These data are tranferred to 
individual test summary sheets as in Figure 5. These sheets are useful for checking 
the ongoing capabilities of the process. 


Tests 

Method: 

Control: 


STANDARD 

DATA FROM MEAN DEVIATION RANGE LIMIT 



FIGURE 5 
INTERPRETATION 

One of the first considerations in setting up the range and average charts was 
( 8 ) 

the subgroup size. Duncan' considers nothing more important. The samples on the 
control chart should represent subgroups of output that are as homogeneous as possible. 
They should be such that if assignable causes are present, they will show up in dif- 
ferences between the subgroups rather than in differences between the members of a 
subgroup. We chose two as our subgroup size because this number of test runs are 
performed within a short period of time on each shift. This is the same reason under- 
lying the maintenance of separate charts for the same test done by different methods on 
different shifts. 
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The average chart is the graphic representation of the accuracy of the process. 
Figure 6 shows normal randomization of points around the mean. One or more points 
falling at or beyond the 2 SE limits need investigation. A run of 7 or more points 
going up, down or running on one side of the center line, indicates a trend or shift 
respectively. Figure 7 illustrates a process that is trending. Figure 8 shows a 
process that has shifted. In all instances the plot points are within the control 
limits but are showing process problems or changes. Cycles or other nonrandom patterns 
in data also need investigation. 



FIGURE 6 FIGURE 7 FIGURE 8 

The range is the graphic representation of precision. If the average plot is in 
control but the range is not, there is need for investigation. 

The ANOVA is the summary of variation both within days and between days. 

ADVANTAGES 

There have been arguments pro and con for the use of ANOVA versus control charts. 

We find the combination of the two to best serve our needs. The control charts are 
a visual device. Even personnel with little grasp of the underlying QC principles can 
read them and understand when an out-of-control situation exists. These charts are 
excellent for current process control. 

The ANOVA gives an overall idea of how a process is performing. It gives a maxi- 
mum amount of information on limited amounts of data. This can be useful when con- 
sidering introduction of a new method. 

Approximately 170 individual tests are charted monthly. Computerization of the 
system has shortened chart preparation time considerably. Calculation of ANOVA' s for 
this volumn of tests is an added statistic not possible when done by hand. The origi- 
nal cost of programming and the ongoing cost of keypunching and computer time is well 
offset by the time saved and the documentation produced. Thus, we satisfy ourselves, 
the clinicians, and our accreditating agencies of the quality of the results produced. 
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MOTIVATION 

The need for quality of care evaluations (Medical Audit) and for cost control 
(utilization review) has become of increasing importance in recent years. This is 
especially true because of the increasing role of third party payors especially the 
various governmental programs. Utilization review concentrates primarily on the 
length of stay for various illnesses and the Physician must certify that extension in 
time is based upon patient needs. Each Hospital staff under rules of the Joint 
Commission on Accreditation of Hospitals (1) must have a committee which provides an 
audit function on the length of stay of hospitalized patients. More recently Public 
Law 92-603 (2-3) contains provisions for the establishment of Professional Standards 
Review Organizations, or PSROs, which examine the standard of care provided to 
hospitalized patients. While formalized by Federal Legislation in the 1972 law for 
PSRO Medicine had already taken the lead in studying its own practices by intro- 
spective review and most medical audits of today include useful items not actually 
demanded by the law itself or subsequent regulations. 

THE SYSTEM 

A variety of different audit approaches have been advocated. Despite the 
varying needs all existent systems have common data items. Additional items may be 
unique to one system or need. The MAN/MACHINE Medical Audit system described herein 
combines a number of these data items into one scheme which can then be used for more 
than one purpose. 

Those items needed to accomplish a desired audit include at least the following 
categories: 

1. Demographic Data; 

2. Indications for admission and date; 

3. Diagnoses and date of establishment; 

4. Medical procedures and date performed; 

5. Complications and date of occurrence; 

6. Indications for discharge and the date of discharge; 

7. Disposition of the Patient and destination after discharge; 

§. Detailed charges. 

Some of these data items are handled similarly in a number of approaches 
stretching from completely manual systems all the way to completely computerized 
systems. Many of the data items can only be handled by utilizing coding schemes to 
express otherwise unwieldy or even unintelligible information. Design of coding 
systems, to be useful, must have provided codes for all of the categories which must 
be machine processed. Many existent codes may be used for medical diagnoses and 
complications including HICDA II, ICDA-8 and the Systematized Nomenclature of 
Pathology (SNOP). A new code to be introduced in January 1979 is ICD-9-CM which will 
also provide procedure codes in addition to diagnoses and complications. The 
American Medical Association also has a procedure scheme called Current Procedure 
Terminology IV or CPT IV. (codes are discussed extensively in Ref. #4) The newly 
introduced Systematized Nomenclature of Medicine (5) has been designed to provide 
code numbers to all the required fields necessary to computerized auditing including 
many categories not available in other systems. The multifield concept and 
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thoroughness of SNOMED have given it superiority as the code system of choice for all 
auditing purposes by computers. 


CRITERIA SETS 

The next common step after choosing the data items and choice of the coding 
system for use is to develop appropriate sets of criteria for the various medical 
conditions. By grouping diagnoses into broad categories, each represented by a 
unique criteria set, the complexity and size of the final system is reduced. This 
step involves the creation of medical committees in the various specialties to 
derive suitable criteria each in their own area of expertise. An example of the 
result of this task is the "Inpatient Medical Care Criteria" manual developed by the. 
Pennsylvania Medical Society in conjunction with the Hospital Utilization Project. 
Needless to say, this activity is peer-oriented, performed by medical professionals. 

In addition, it is expected to be an ongoing function, since advances in medical 
science will modify the professional standards to be applied in assessing health care. 
An estimated two hundred criteria sets will cover 90-95% of all admissions in a 
general acute hospital. 

Although the criteria sets are formulated primarily for quality control, they 
may also include billing information to determine the economic impact of alternate 
treatments. Three components make up the established criteria set: 

1. List of Diagnoses which will fall under the set; 

2. List of Audit Items which are to be collected and assessed in order to 
perform the audit; 

3. A representation of the valid combinations representing acceptable health 
care. 

In the Medical Audit System, (1) and (2) are represented by codes which can be 
translated back into English as necessary. The valid combinations of audit item 
values are represented through Acceptability Masks, each mask standing for a unique 
combination of audit items which are considered satisfactory for treatment under 
the criteria set. 

Determining the techniques to be used in collecting data values for the audit 
is step four in design of the Medical Audit System. As much as possible, existing 
automated sources in the institution should be relied upon. This ensures timeliness, 
accuracy, and consistency of data, as well as reducing redundancy of collection 
procedures. Few hospitals, however, will have all the necessary information already 
available in machine-readable form, however, and none will have it formatted and 
structured in the ideal fashion for audit purposes. Final decisions in this area will 
also be affected by future plans of the hospital toward automating medical data. 

As the values of audit items for a case are collected, an Audit Compliance 
Mask is built up for each diagnosis. This mask, created 1 to make it compatible with 
the Acceptability Masks for the criteria set covering the diagnosis, indicates 
whether or not each audit item was satisfactorily performed. 

Lastly, and most importantly, the audit procedure itself must be formulated. 

The basic assumption made in establishing a medical audit system,' that all cases 
treated at a hospital are subject to review by suitably qualified peers of the health 
care providers, is extended in the Man/Machine Medical Audit System. Rather than 
requiring every case to be manually audited the criteria set information is automated 
and the majority of cases are handled by a machine audit. Then the only cases which 
must be man-audited are those for which suitable criteria sets have not been 
formulated, and those which, for some reason, fail to pass an established automated 
criteria set. This basic system concept solves the primary problem seen with 
medical auditing in a strictly manual mode: the vast amount of professional time 
required to accomplish the audits. 

The audit algorithm itself should be simple, universal, and amenable to 
manual performance. In the Man/Machine Medical Audit System, the algorithm consists 
of comparing the Audit Compliance Mask built up for each diagnosis in the case to the 
set of possible Acceptability Masks defined for the relevant criteria set. This 
comparison is done using simple logical operations built into all computer systems 
and available in most programming languages. The result is a definite pass or 
fail, which can be displayed with relevant information about the case, as desired by 
tne committee in charge of the review process. 
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Medical auditing can be ongoing or retrospective. For the purpose of compiling 
information for federal P.S.R.O. use, a retrospective audit is sufficient. Depending 
on the degree of hospital commitment, however, the system should be evolutionary, 
and capable of being used for continual auditing during the patient's stay. 

DATA ITEMS 


The following twenty data items include all of the above listed categories 
deemed necessary to perform a useful audit: 


#001 

Personal Identification Code 



#002 

Date of Birth (mo-day-yr) 



#003 

Sex (M-F) 



#004 

Race (Collected after admission) 



#005 

Residence (zip) 



#006 

Hospital Identifier 



#007 

Admission Date (mo-day-yr) 



#008 

Discharge Date (mo-day-yr) 



#009 

Attending Physician (Regional #) 

#109-209, 

etc. consultant, etc. 

#010 

Operating Physician (Regional #) 

#110-210, 

etc. additional surgeons 

#011 

Diagnoses (Any code system) 

#111-211, 

etc. final diagnoses 

#012 

Procedures - Dates 

#112-212, 

etc. extra procedures 

#013 

Disposition of Patient 



#014 

Expected Source of Payment 




PSRO Additions Needed to be in Compliance 

#015 Detailed Charges - #015 - Lab 

- #115 - XRay 

- #215 - Pharmacy 

- #315 - Other Services 

#016 Class of Admission 


Audit Additions Which Allow the Quality of Care to be Better Assessed 


#017 

Comp! ications 

#117-217-317, etc. additional 
complications 

#018 

Indications for Admission 


#019 

Indications for Discharge 


#020 

Audit Diagnostic Class (200) 



By taking various combinations of data items various questions can be answered. 
Length of stay is based upon comparisons of admission and discharge dates. While 
this appears to be relatively simple to do visually it is also a good example of 
data which probably already exists in machine readable and processable form in most 
Hospitals so that the computer can easily be programmed to go the extra step and 
provide an actual automatic calculation and printing of the result. 


Example: Medical Audit for Appendicitis 


Appendicitis is a surgical condition which may be logically analyzed as 
might be derived from a typical record. 


A) Indications for Admission 


1) Typical History including: Nausea and Vomiting 

Anorexia 

Right Lower Quadrant Pain 


F-61640 

F-60014 

T-Y4120 F-82609 


2) Typical Physical Findings: 


Rebound Tenderness 
WBC > 11,000 
Fever (Oral) > 100 


F -8261 0 

T-2945 > 11000 
F-03003 > 100 



B ) Type of Admission 

1) Emergency Admission Acceptable P-0030 

2) Elective Admission Acceptable P-0021 

C ) Pathologic Diagnosis 

1) Positive for Acute Appendicitis T-66 M-4 

- if Positive A1 and A2 above may be Negative and pass 
the audit. 

- if Negative Pathology the A1 and A2 must be present 
to pass the audit. 


D) Pre-Op Hospital Services Required 


1) Complete Blood Count Done P-2814 

2) Urinalysis Done P-4150 

- if these were not done the case fails the Audit 


E ) Pre-Op Hospital Services Acceptable for this Diagnosis 


1) Fasting Blood Sugar 

2) Urea Nitrogen or Creatinine 

3) EKG if over 35 years old 

4) Chest X-ray 

5) X-ray of Abdomen 


T-OX P-4020 
P-4020 F -1 0460 or 
F- 11 390 
P-7110 

T-Y21 P-X000 
T-Y41 P-X000 


- Procedures other than above must be explained before payment is made. 

F) Expected Length of Stay 


1) One Day Pre-Operative (Calculated) 

2) Post Operative up to 7 days (Calculated) 


- These are calculated from admission, operative and discharge dates. 

G ) Complications Extending the Length of Stay 


1) Peritonitis T-Y44 M-4 

2) Wound Infection M-14010 

3) Pneumonia T-28 M-4 

4) Phlebitis T-48-49 M-4 

5) Abscess M-4174 

6) Urinary Tract Infection T-69 M-4 


- Complications automatically extend acceptable Length of Stay. 

H ) Indications for Discharge 


1) Patient Ambulatory P-0105 

2) Afebrile < 99° for 24 hours F-03001 

3) Full Diet E-X030 

4) Alive F-00101 


- Acceptable for discharge if all the above are present. 

N.B. Since SNOMED is carefully designed as a hierarchical code in which each digit 
is significant the above fields are truncated as the computer search would be. 
Example: M-4 is a totally inclusive search algorithm which contains any and all 
the variable ways of specifically coding an inflammatory process. Five full 
digits are available such as: Multifocal Fibrocaseous Inflammation M-45273. 

This descriptive term would still satisfy the audit criterion of "inflammation" 
and would be picked up correctly by the simple search algorithm "M4". Note 
that in the Procedure (P) and the Function (F) fields the full numbers 
available are usually necessary to adequately specify that P or F concept. 



GENERAL CONCEPT OF THE AUDIT SYSTEM 


The general approach of this Man/Machine Audit is to develop a series of 
criteria which will cause a case to be acceptable to the audit committee. This 
"criteria set" must include some flexibility to account for any acceptable alterna- 
tives for cases. These are stored item by item in a set identified with each 
disease. 

Whan an audit is to be performed the system collects the audit items on a 
specific case in proper order and compares these item by item with the "criteria set 
for this diagnosis. If each item is acceptable according to the criteria set the 
case passes the machine audit". 

If any single item fails to match and to prove acceptability then the case 
fails the machine audit. The criteria item or items are automatically listed to 
direct the attention of the audit committee to that item for their visual review. 

It must be stated that the committee establishes the criteria set it will 
accept in the first place. Thus it may be said that any case passing the machine 
audit has in truth really passed the manual (committee) audit. The goal of the 
committee is to select a criteria set which will minimize the number of Machine 
rejected cases and yet criteria will be defendable to other physicians as reasonable 
and proper. It must be remembered also that "failing" the machine audit only means 
that it must be individually scanned by the committee not that there will be 
predictable failure of the manual audit. The variability of cases within acceptable 
practice limits is quite broad. It is a reasonable goal if the committee can raise 
the audit criteria set to the level that 95% of cases can be automatically handled 
with only 5% of the cases requiring special discussion. 

SUMMARY 

The general need for evaluation of the quality of medical care has been 
discussed. Most of the data items needed to provide the audit function under 
control of the Medical Staff are gathered routinely by the hospital's own medical 
record keeping and accounting functions. The additional items required of a medical 
nature can be accumulated as a byproduct of other medical activity. It is explained 
that the Systematized Nomenclature of Medicine (SNOMED) is the only coding system 
sufficiently detailed to allow the computer to compare actual patient care with 
predetermined norms established by the Medical Staff and stored in the computer. 
Individual case analyses can then be gathered into statistical groupings for 
disease by disease analysis. The automated machine audit is an extension of a 
manual audit which would otherwise have to be performed on each case. Early 
experience suggests that approximately one case in twenty, or less, will require 
committee inspection. 
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APPENDIX 


Example of Single Case Audit which Passed the Audit 


MEDICAL AUDIT SYSTEM REVIEW OF FIRMIN DESLOGE HOSPITAL 


CHART NUMBER 69-1091 
HOSPITAL CODE 01 
BIRTH DATE 8/5/50 
ZIP CODE 63113 
SEX F RACE 2 

PAYMENT SOURCES 6 

PRIMARY ATTENDING PHYSICIAN 

PRIMARY OPERATING PHYSICIAN 

SECONDARY OPERATING PHYSICIAN 


ADMISSION TYPE 2 
REFERRAL SOURCE 1 
ADMISSION DATE 2/3/78 
DISCHARGE DATE 2/9/78 
DISCHARGE STATUS 1 
DESTINATION 

9426 SPAFFORD, 

9426 SPAFFORD, 

9754 WILLMAN, VALLEE L 


DIAGNOSIS 0004510000 PRIMARY APPENDECTOMY 
ESTABLISHMENT DATE 2/3/78 OPERATION DATE 2/9/78 
TERMINATION DATE 2/9/78 

ITEM VALUE RANGE 

2 NO COMPLICATIONS 

3 AFEBRILE (LT 99.6 FOR 24 HOURS) 

4 AMBULATORY 

5 FULL DIET 

6 ALIVE 

7 TOTAL LENGTH OF STAY 6 27 

8 POST-OPERATIVE LENGTH OF STAY 0 07 

31 HX DESCRIBES RIGHT LOWER QUADRANT PAIN 

32 HX DESCRIBES REBOUND TENDERNESS 

33 HX DESCRIBES NAUSEA, VOMITING, ANOREXIA 


LCS 710:70:880 



QUALITY CONTROL IN A HEALTH INFORMATION SYSTEM 


Roland J. Loup, PhD, Manager, Quality Control 
Mary K. Murphy, Associate Quality Control Analyst 
Cynthia L. Russell, Associate Quality Control Analyst 
Commission on Professional and Hospital Activities 
Ann Arbor, Michigan 48105 

INTRODUCTION 

Awareness of the need for accurate data in health information systems has 
increased in recent years. New emphasis is being placed on the quality of the 
coding and abstracting of information obtained from patient medical records, 
and on the adequacy of controls and audit procedures used in data processing. 

Data quality has long been the concern of the Commission on Professional and 
Hospital Activities (CPHA)*. The PAS System, centered around CPHA’s main pro- 
gram, the Professional Activity Study (PAS) , is a medical record information 
system that currently processes over 17 million discharge abstracts per year 
from over 2,200 U.S. and Canadian hospitals. Hospitals that participate in the 
PAS System record on one-page preprinted documents the data abstracted from the 
medical record of each patient and mail these documents to CPHA where they are 
computer processed. Hospitals receive routine and special reports as part of 
their participation in this system. 

CPHA’s data quality control activities in the PAS System can be grouped into 
two major categories: 

1. that related to the quality of coding and abstracting done by the hospitals, and 

2. that related to the processing of the data from approximately 65,000 
abstracts received each day. In particular, specific attention is paid to 
the quality control of data accuracy, completeness (complete in that the 
system contains all the records it should), security, confidentiality, and 
timeliness. 

This paper addresses CPHA's quality control activities in both areas. 

QUALITY CONTROL OF CODING AND ABSTRACTING 

Traditionally, medical record department personnel coded a patient’s diagnoses 
and operations as a means of retrieving the records according to their subject 
matter. More and more, coded data are becoming the foundation for statistical 
reports which are used to make decisions in the management of the health care 
industry. Today the uses of statistics gathered from the information in medical 
records are vast. For example, coded data are often used to 

assess the quality of patient care, 
set priorities for improvement of patient care, 
assess the utilization of facilities, and 
determine reimbursement rates for insurance claims. 

The new applications of these data have increased the importance of coding 
accuracy. ' And the users, in order to make well-informed decisions, need quanti- 
tative measures of the accuracy of the data upon which they are basing those 
decisions . 


* located in Ann Arbor, Michigan, CPHA is a nonprofit, noncommercial, and 
nongovernmental research and education center dedicated to the improvement of 
hospital and medical care. 
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The first significant documentation of coding accuracy was obtained through a 
study conducted by the Institute of Medicine, a division of the National Academy of 
Sciences. In that study, conducted in 1975 and published in 1977, the level of 
reliability in the coding of principal diagnosis (the final diagnosis most 
responsible for occasioning the patient's admission to the hospital) was 65%. 

This result makes clear the need for both raising coding accuracy and developing 
quality control procedures to guarantee that the coding of data is at a level of 
excellence. 

CPHA recognizes that the first step in producing reliable statistical reports 
is to ensure that the coded and abstracted data coming into the PAS System are as 
accurate as possible. The Commission, therefore, shares with the hospital the 
responsibility for acquiring quality data and meets this responsibility by pro- 
viding an easy-to-use abstract, detailed reference materials, training and 
consultation, and methods for evaluating coding and abstracting performance. 

PAS Case Abstract 


Each hospital in the PAS System is supplied with abstracts that have been 
carefully designed to promote the accurate recording of data. The abstract is a 
computer-scannable document with a mixture of bubble mark-offs and hand-formed 
characters . 

Training and Consultation 

At the time a hospital begins participation in the PAS System, it enters into 
an intensive training program. A CPHA representative phones the chief of the 
medical record department and schedules an in-person visit to the hospital. 

Together they review all. the reference materials sent to the hospital including 
the PAS System Manual, ' a manual of over 100 pages containing detailed 
instructions and illustrations for completing each abstract item, and a pro- 
grammed instruction manual'" for coding from the Hospital Adaptation of the 
International Classification of Diseases , Second Edition ( H-ICDA-2 ) C5T~ used in 
converting all diagnoses and operations into numeric codes. 

CPHA then assigns a coding specialist and a liaison representative to train 
and consult with each hospital. During the training period the hospital enters on 
the abstract both H-ICDA-2 codes and the English terminology for all diagnoses and 
operations. The abstracts are sent to CPHA and checked for accuracy by the coding 
specialist. During a typical month, CPHA' s' coding specialists examine more than 
40,000 abstracts submitted by training hospitals. In addition, telephone confer- 
ences take place to review coding and abstracting principles. 

When a hospital has completed abstracting all the discharges for its first 
month's data, its PAS System reports are generated. One set is sent to the 
hospital and one set is produced for the hospital’s liaison representative who 
checks the reports for inconsistencies that could indicate systematic coding or 
abstracting problems. A telephone conference is then held with the personnel at 
the hospital to review any problems. At this time a report review is also con- 
ducted by CPHA staff to familiarize medical record personnel, administrators, and 
medical staff with the kinds of data they will receive routinely and the ways in 
which these data can be effectively used by the hospital. 

The training period ends when the hospital has reached an acceptable level of 
coding accuracy and has demonstrated a good understanding of the PAS System reports. 
The liaison representative continues to act as a consultant to the hospital in 
resolving any future questions or problems. If the medical record department feels 
retraining is in order, for example, because of personnel changes, it may request 
to be returned to training status. 

Evaluation of Coding and Abstracting Performance 

As mentioned above, the reports themselves are a valuable tool for determining 
data accuracy and completeness. The more the data are used, the higher the level of 
accuracy. To stimulate report use CPHA continually conducts education sessions that 
encourage better quality control of input and provide practical applications of PAS 
System data. Hospitals are urged to review their reports and submit the necessary 
corrections if any errors are discovered. 



CPHA has developed a new method of quality control for assuring coding 
accuracy that applies principles long used in industry. This quality control 
method is designed to help detect and correct procedural or systematic errors in 
the coding process. 

The method is based on the following five steps in the quality control 
process : 

1. Establish criteria (set standards for defined parameters). 

2. Measure performance. 

3. Analyze deviations. 

4. Take action. 

5. Follow up. 

CPHA has published a monograph entitled Quality Control of Diagnosis and 
Procedure Coding ^ ' which describes the quality control method, presents specific 
criteria for assessing coding accuracy, and provides a means for applying the 
method, the Quality Control Worksheet for Coding. The worksheet guides the user 
through the five steps of the process and provides for recording the results of 
each step. Upon completing such a quality control study, the medical record 
department supervisor will know not only whether errors are occurring, but 
also why they are occurring and what plans can be implemented to correct poor 
performance. 

The use of a quality control process employing objective criteria, with 
statistical standards for accuracy against which actual performance can be measured, 
represents an effective and efficient method for controlling and improving coding 
accuracy. More importantly, a mechanism is provided to detect and ultimately 
eliminate the causes of error. 

QUALITY CONTROL IN PROCESSING PAS DATA 

Once the hospital submits its patient information to CPHA for processing, 
numerous control techniques are applied throughout the PAS System to maintain data 
quality and produce reliable statistical reports. 

Processing of Batches 


As a method of document control, abstracts are usually batched into groups of 
50 or 100 by the hospital. All batches contain abstracts only for patients dis- 
charged from the hospital in the same month. Accompanying every batch is a batch 
cover sheet containing the hospital’s identifying number, batch number, discharge 
month, and the number of abstracts within the batch. 

Upon receipt of a batch at CPHA, the first and last abstracts are verified 
against the information on the batch cover sheet. The batches are inspected for 
the presence of other documents, such as correspondence inadvertently inter- 
spersed with the abstracts. Document control begins immediately in that all the 
information contained on the batch cover sheet is entered into the computer the 
day the batch is received. 

The hospital continually submits batches of abstracts as they are completed. 
The medical record department maintains a control report by discharge month for 
batches and abstracts mailed and a stub from each abstract with the patient’s 
number, batch number, and page number within the batch. When the hospital sends 
the last batch for a month, it also encloses a form notifying CPHA that abstracts 
have been completed for all patients discharged in that particular month. The 
’’closing" hospital is given priority in processing. 

Optical Scanning 

After a batch of abstracts has been inspected, it moves to the next phase in 
the PAS System: being electronically read by an optical scanner and stored on 
computer media. All information on the batch cover sheet and on the abstracts 
is scanned. The entire batch is rejected if there is any discrepancy between the 
information read from the batch cover sheet and the information originally entered 
into the computer the day the batch was received. When the abstracts are read, 
they are also counted. After scanning is complete, a control report is produced 



indicating the total number of abstracts recorded on the cover sheet and the total 
number of abstracts read by the scanner. If batches are received out of sequence, 
a second control report will indicate a missing batch. 

As a continual check on scanner accuracy in reading bubble mark-offs, a batch 
of test abstracts is scanned every two hours. Test output is compared against 
the actual entries on the abstracts. If there is a discrepancy between the two, 
the abstracts and the data tape of all batches run since the last scan test are 
held, the scanner is realigned and the test batch rerun. When the test output 
and the abstracts are in agreement, the held batches are rescanned and a 
corrected tape is produced. 

Scan Verification 


Due to individual variations in handwriting, the hand-formed characters 
scanned from the abstract and the batch cover sheet are independently re-entered 
into the computer system. Data entry operators key-verify approximately 65,000 
abstracts a day using cathode ray tubes (CRT) on-line with CPHA's main computer. 

The operator enters the data directly from the abstract. This information is 
displayed on the screen immediately in front of the verifier. If a difference 
occurs between the keyed data and the scanned data, the screen flashes back that 
a discrepancy has taken place. The operator re-enters the recorded value from the 
abstract to resolve the problem. If the verifier is unsure of the actual value, 
the abstract is flagged for further review. If necessary, the hospital is called 
to obtain the correct information. 

The operator also verifies the data on the batch cover sheet. Again, if there 
is a count problem, the verifier makes the final determination as to the correct 
number. The hospital is always notified of any count changes, so that it can updat 
its internal control counts and forward any missing abstracts. 

Edit 


After verification, the data from each batch automatically go through an 
editing process. The primary function of the edit is to detect any errors intro- 
duced during the verification process, for example, the presence of alphabetic 
characters in fields authorized for only numeric data. 

Messages are printed for each patient record with an error. The error 
message is matched against the original abstract. If necessary, the computer 
record is corrected and the batch is again edited. If the error was not caused 
by scanning or by verifying the document, the error is left and the next step in 
the quality control procedure, the audit, will generate an audit error message 
which is sent to the hospital. 

Audit 


When editing is completed, the patient record undergoes an extensive quality 
check called the audit. Over 300 data consistency and completeness checks are made 
for example: 

1. completeness of all basic patient information, such as admission and 
discharge dates, 

2. agreement between sex-specific diagnosis codes and the recorded sex, and 

3. reasonableness of certain data items, such as patient weight of over 400 
pounds. 

The data are then transferred to a working computer file. All the hospital's 
patient records are now easily accessible ("on-line") . 

Another important result of the audit process is the computer- generated 
audit error message. 


Computer-Generated Audit Error Messages 


An audit error message is produced every time the patient record fails an 
audit check. Approximately 5% of all abstracts generate audit error messages. 

A data control technician compares the error found to the original abstract 
item to determine if the error was introduced during processing. If it was, the 
correct information is entered into the system. Coding problems detected by the 
audit are forwarded to CPHA’s coding specialists. Errors not correctable at CPHA 
are mailed to the hospital. To minimize the possibility of additional problems, 
the audit error message contains all the information recorded on the patient 
abstract. The hospital enters the correct data directly on that form before 
returning it to CPHA. 

Corrections and Supplemental Information 

A hospital can also generate its own corrections by adding information 
omitted from the original abstract and by changing or deleting data currently 
recorded on the abstract. Information which would not fit on the abstract, such 
as additional diagnoses, operations, or optional data (data defined by the 
hospital) is submitted as supplemental information to CPHA. 

Corrections and supplemental information are entered by a CRT onto the working 
file. As the data are being entered, an abbreviated version of the audit takes 
place on the items being applied. Corrections are not accepted by the system if 
this check detects a problem. An error message flashes on the screen. The operator 
determines if a true error exists and, if so, copies the message onto the 
correction or supplemental form, and returns it to the hospital for action. 

The hospital may submit corrections and supplemental information until all 
its abstracts have been processed for a calendar semiannual period. 

PAS System Control Files 


Several control files oversee the entire PAS System: the batch and report 
control files, and the hospital identification file. The batch control file keeps 
track of each batch as it goes through the various processing steps. A batch 
cannot go on to the next step unless all prior processing steps have been completed. 
The report control file indicates when a hospital’s data are ready for report 
production. 

The hospital identification file sets hospital-specific values used in the 
audit, such as definitions of the hospital’s clinical services and optional data. 

This file also governs which reports are generated and their frequency. 

The hospital identification file as well as the batch and report control files 
are accessible on the CRT for reference. These displays inform authorized personnel 
as to the status of an individual patient record, a batch, or a report. With this 
information readily available, it is used for data quality investigations. 

Report Controls 

Standard PAS reports are produced for monthly and semiannual time periods. When 
the hospital notifies CPHA that abstracting is finished for a period, the "closing" 
hospital is given priority throughout the PAS System. Any outstanding batches are 
scanned; all corrections and supplemental information that have been received are 
applied; the data are edited and audited. Batch and abstract counts on the working 
file are compared to the corresponding information on the hospital’s closing notice. 
No reports can be processed until these counts are reconciled. 

Once the working file has been updated and all counts justified, the reports 
are produced. Generally CPHA processes and mails monthly reports back to the 
hospital within 48 hours after receipt of the closing notification. Once the 
semiannual reports have been processed, the data are transferred to permanent 
file storage and become a part of over 150 million hospitalizations contained in 
the PAS data library. 



Report Inspection 


All PAS reports are inspected for usability, accuracy, and internal con- 
sistency prior to mailing. If printing is smudged, off-line, or omitted, or any 
pages are torn, wrinkled, or missing, the reports are run again. Independent 
control reports are generated indicating total patiencs processed for the hospital's 
report. These control counts are compared against the actual totals found on the 
hospital's report. Any detected problems are referred to the Quality Control 
Department for research and resolution. 

Quality Control Department 

Because all of CPHA's data processing is contracted to Electronic Data 
Systems (EDS), the role of the Commission’s Quality Control Department (QCD) is 
essential in assuring the quality of data being processed. The QCD participates 
in setting system performance standards, monitoring the performance in relation 
to those standards, investigating PAS System problems when they arise, making 
recommendations to management when performance is below the standards, and 
developing new ways to improve the quality of data. 

To maintain control of all PAS System changes, the QCD has designed data 
files which are used for testing all new subsystems and any program changes. 

This final testing ensures that the entire PAS System continues to operate as 
intended. 

PAS System Security 

Security measures are employed within the PAS System not only to guard against 
theft or improper access to the data, but also to provide recovery capability for 
lost or damaged data. The problems of maintaining data security are compounded 
by the complexity of the PAS System and the large volume of data moving through 
that system. Security procedures pertaining to recovery capability include main- 
taining copies of all data files and all programs and control files used in each 
processing step. These serve as a back-up and enable CPHA to automatically recover 
data marred by a software or hardware malfunction. Back-up programs and control 
files critical to system operation are updated weekly and stored at an offsite 
location. Twenty-four hour physical plant security is enforced. In addition, there 
is limited access to specific areas within the building, such as the computer room 
and the tape vaults containing the back-up files. 

Confidentiality of PAS Data 

The usefulness of PAS data for research and planning has prompted numerous 
requests for studies by member hospitals and outside organizations. CPHA 
recognizes the potential of PAS data while maintaining a strong commitment to the 
strict confidentiality of all data submitted by PAS hospitals. The contract 
between CPHA and client hospitals stipulates that CPHA will not divulge the 
identity of a hospital included in any report, publication, or study. If the 
identity of a hospital is required by a researcher, or if it can be deduced from 
the uniqueness of the data or the small sample of hospitals studied, a three-way 
written agreement between the hospital, the researcher, and CPHA must be made. 

The identity of physicians and patients cannot be divulged since CPHA is furnished 
only with hospital-assigned codes as identifiers. 

It is emphasized to every CPHA and EDS employee that he is dealing with infor- 
mation of a confidential nature. All CPHA employees and all EDS employees 
assigned to CPHA's account must sign a policy statement acknowledging agreement 
on this subject. Under no circumstances may an employee of CPHA release any 
confidential information unless authorized by the president of CPHA. The 
identification of any hospital, physician, or patient by name or number in 
relation to any data passing through the employee's hands is forbidden. 



Conclusion 


CP HA is committed to having high quality data in the PAS System. Fulfilling 
this commitment requires having not only ongoing, systematic quality control 
activities, but also creative applications of quality control principles for the 
accuracy, completeness, security, confidentiality, and timeliness of the data 
in the system. 
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A BUSINESS SYSTEMS APPROACH TO GMP'S 
A Guide to the Small and not so Small Medical Device Manufacturer 


Ervin F. Taylor 

Corporate Director Quality Control 
American Cystoscope Makers, Inc. 
Stamford, Connecticut 06902 


AN INTRODUCTORY SCENARIO 

Company X has been manufacturing medical devices for some time. 
Customers are generally satisfied with their products. There have been 
some complaints, satisfactorily resolved. There also have been a few 
minor product liability actions, and limited product recalls. Company 
X is making a reasonable profit. 

Enter the Medical Device Good Manufacturing Practices (GMP). 
Company X organizes to comply. After listening to a number of experts 
on the ways of compliance, they develop a comprehensive GMP compliance 
system. 

Not unexpectedly, costs increase and profits decrease. However, 
unexpectedly, customer complaints also increase! Then, liability 
actions increase and insurance costs rise. FDA inspectors find non- 
compliance ! 

Company X adds resources to cope with the myriad of problems. 

Costs increase further — profits decrease even more. The spiral con- 
tinues. The business fails. 

WHY? 

The answer is simple. Company X did not attend to their own 
business FIRST. They directed their resources toward GMP compliance, 
but lost the business. 

THE STATE OF MEDICAL DEVICE BUSINESSES AND GMP'S 

Most medical devices from most manufacturers ARE safe, effective 
and meet the customers' needs. A little reflection will demonstrate 
the verity of this statement. Examine first the medical devices you 
know best — your own. The statement certainly applies. You may have 
a few problem devices, but you are working to solve those problems. 

Look now at the medical devices you know next best — your competitors ' . 
You will find that the same situation holds true; and for a very good 
reason. Both you and your competitors are in business to earn a pro- 
fit for the business owners — the stockholders. If most of your 
products were not safe and effective, and did not meet your customers' 
needs, your business would not long survive. 

Since most medical devices from most manufacturers are safe and 
effective, and meet the customers' needs, it follows that most manu- 
facturers are already following good manufacturing practices. They 
are doing many things right I You are doing many things right! To be 
sure, many of your good practices may be an unstructured and undocu- 
mented way that you do business; however, these good manufacturing 
practices do exist. They are responsible in no small measure for the 
success of your business. 

When some businessmen examined the FDA proposed GMP's for medical 
devices, they stated they should be called "good business practices" 
instead of "good manufacturing practices” since the scope of the 
proposal extends far beyond fabricating a medical device. Design, 
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production control, maintenance, industrial relations, quality control, 
packaging, and distribution are some of the other traditional business 
functions affected. Make no mistake, GMP's are quality control system 
requirements. They are the requirements for that business system 
which controls the quality of medical devices. GMP's are not, however, 
necessarily synonymous with quality control organization. While there 
are organizational aspects to the GMP, the scope of the requirements 
includes far more than that usually within the purview of the tradi- 
tional quality control organization 

A BUSINESS DEFINITION OF "QUALITY" 

Before one can address the subject of a business systems approach 
to GMP's, a full understanding of the meaning of the term "quality" 
in a business sense is essential. Almost every text on the subject 
of quality control includes (usually in Chapter 1) a definition of 
"quality". Appropriate as these definitions may be, they are from a 
professional technology rather than a business point-of-view . They 
suffer from the "zero defects" or perfection syndrome. While there 
may be aspects of the business where striving for perfection is pro- 
ductive and profitable; there are many other areas where the converse 
is true. 

QUALITY IS THE DEGREE TO WHICH THE ACTUAL MEETS THE EXPECTED. This 
is a business definition of "quality". Note that this definition is 
sufficiently broad to include every aspect of the business. Note also 
that "quality" can be both negative or positive in degree. 

When the actual is less than the expected, the quality is 
too low. 

When the actual equals the expected, the quality is optimum. 

When the actual exceeds the expected, the quality is too 
high. 

For example, suppose that a device performance will meet the cus- 
tomer's expectations if a certain component tolerance is met. Suppose 
further that an over-zealous design engineer puts only half this tol- 
erance on the drawing for a margin of safety. Already, the actual 
(engineering drawing) exceeds the expected (customer requirement) 
and the quality is too high. In one case, a production worker, 
equating a tight tolerance for a need for perfection, takes extra 
time and effort and uses only a portion of the allowed tolerance. Again, 
the actual (component) exceeds the expected (tolerance) and the 
quality is too high. In another case, a production worker may be- 
lieve that the tolerance is unrealistic, and produces components 
that are out-of-tolerance and are rejected by the inspector. In this 
case the actual (component) is less than the expected (tolerance) and 
the quality is too low. 

Consider the excess costs — and the profit loss — in these examples. 
The situation would be even worse if the rejected components (actual) 
did in fact meet the customer's requirement (expected). It is for 
this very reason that it is important from a business viewpoint to 
define quality in a manner which permits control against being too 
low or too high. The general relationship between the actual, the 
expected and the relative business risk is depicted in Fig. 1. 



FIGURE 1 
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Note that when the actual is well below the expected, the relative 
business risk is quite high. This is the classic case of too low qual- 
ity. However, the relative business risk increases when the actual 
exceeds the expected and the quality is too high. The relative business 
risk is a minimum when the actual equals the expected. 

The entire process of business management (and medical device 
businesses are no exception) involves determining the expected and 
controlling the actual to meet the expected. This is the substance 
of the business systems approach to GMP's. It is based on this author's 
fundamental belief that if you operate your business in such a way as 
to maximize long term profits, you will produce medical devices which 
will be safe and effective, meet your customers' needs, meet the regu- 
latory requirements and minimize product liability. 

THE BUSINESS SYSTEMS APPROACH TO GMP'S— A 10 STEP PROGRAM 

You best know your own business — its personality, its resources, 
its constraints. You know your business better than your customers 
know it, better than other outsiders know it, and better than FDA 
inspectors know it. Your GMP must be specifically tailored to your 
business, and it will be for your business only . It would be utter 
business folly for you to attempt to overlay someone else's GMP onto 
your business system. 

The following 10 steps are designed to assist you in discovering 
your own good manufacturing practices, and to enable you to develop 
your own GMP compliance system at a minimum cost. During the process 
you may find that you need to alter your business system to meet your 
own business needs. Once this is accomplished you will very likely 
find that you then meet the GMP requirements. 

STEP 1 — Determine your present business system. 

Your present business system consists of defining the 
expected and controlling the actuals to meet the expected 
at many levels. How do you determine customer expecta- 
tions? Do you exercise any control over these expectations? 
How do you translate customer expectations into engineering 
requirements? What are the controls on design? How do 
you translate design requirements into work orders, tool- 
ing, purchases, inspection and labeling? What are the 
controls? How do you establish the expectations for 
facilities, capital equipment, expense tooling and 
gaging, personnel selection, personnel training, and 
facility equipment maintenance? What are the controls? 

To assist you in the task of determining your present 
business system, the author has available on request a 
"Checklist of Business Systems for the Control of Quality". 
This checklist consists of more than one hundred questions 
to determine those business systems which you may have or 
need. All of the questions may not apply to your specific 
business. However, many of the questions will prompt 
additional questions which will emerge from your inti- 
mate knowledge of your business. 

STEP 2 — Diagram your present business system. 

Once you have determined your business system, diagram 
it. You can use the full range of symbols which the 
systems engineers are fond of using; however, in most 
cases, a simple block flow diagram will suffice. There 
are several reasons for diagraming your systems. It will: 

1. Help you to understand your system. 

2. Help others to understand the system. 

3. Facilitate the detection of redundancies 
and gaps . 


4. Provide a reference baseline for your system 
audits . 

5. Provide a reference baseline for system 
changes when products, resources and con- 
straints change. 

6. Simplify the drafting of operating procedures 
which will be your GMP "manual". 

STEP 3--Document business system requirements (policies) to 
meet your business needs. 

Examine your business system diagrams. Ask the question, 
does the business system meet ray business requirements? 

To answer this question requires that your business re- 
quirements be known. These requirements should be 
documented and will form the policy statements for your 
business. Such policy statements are important to busi- 
ness management for they define requirements — what must 
be done — rather than the detailed procedures — how it is 
to be done. The policies you establish will include 
most, if not all the GMP requirements. Policies tend to 
be static, changing only when the fundamental business 
changes. Policies are especially important for multi- 
plant operations. The operating procedures at each 
plant may vary according to facilities and other 
resources, and constraints. However, each plant can 
meet a common policy with differing procedures. 

STEP 4 — Evaluate and modify your system to meet your business 
policy . 

During Step 3 you may have found that your business 
system has excesses or deficiencies with respect to 
your business requirements — your business policies. 

Adjust your business system so that it meets your 
own business policies. You will find that your busi- 
ness system diagram will facilitate changes in a 
manner avoiding redundancies and gaps, and providing 
the links to other business systems. 

STEP 5 — Adjust the business system to GMP requirements. 

Only after you have a business system design which meets 
your own business needs should you look to the GMP. If 
you have completed the previous four steps satisfactorily, 
you will probably have either no adjustment or only a 
minor adjustment to the business system to meet the GMP.' 

STEP 6 — Document your business system technique (procedures). 

Your operating procedures must be documented so that those 
who are responsible for that portion of the business can 
perform the tasks consistently and in a manner which you 
have determined to be most effective and efficient. These 
operating procedures will become your own GMP compliance 
manual. It, along with your policies, is the primary 
document against which an FDA inspector will audit your 
business for GMP compliance. The procedure writing task 
is almost routine when it is done from the previously 
diagramed business system. Include the flow diagram as 
a part of the procedure. 

STEP 7 — Design forms for information retreival. 

Business systems seem to require considerable paperwork. 
Many forms are designed to collect information. Both 
blank and completed sample forms should be included in 
the appropriate procedures to assure proper completion 
and use. Design your forms for the efficient and accu- 



rate retrieval of information. By using this approach 
you will tend to assure that useless and costly informa- 
tion is not collected. Forms designed for information 
retrieval are also usually forms in which it is easier 
to record accurate and legible information. Of course, 
if there is a conflict between ease of information entry 
and ease of information retrieval, suitable trade-offs 
predicated upon business risks should be made. 

STEP 8 — Collect data for use, not storage. 

Businesses collect vast amounts of quantitative data, 
much of which ends up in filing cabinets or computer 
files and is never analyzed for any useful business 
purpose. The storage costs of such data is staggering. 

Before you require the collection of data, make sure 
that you have a specific business purpose for that 
data. The analytic techniques or other business pur- 
poses should be defined in the procedure. Data which 
is "nice to have" or duplicate data is seldom justifiable 
or profitable. 

STEP 9 — Test and adjust the business system. 

Even if you have made only minor modifications to your 
present business system, it is good business practice 
to try it out on a limited basis before a full-scale 
implementation. A limited trial will often reveal 
errors and inconsistencies which are easily rectified 
at that stage. This is especially true of information 
and data collection forms. It is usually much more 
costly to alter a system which has been fully applied. 

Be sure to include any adjustments you may make as a 
result of the trial application in your business system 
diagram. 

STEP 10 — Audit and adjust the business system. 

Business systems are dynamic. They change with new or 
different people among other factors. Subtle changes 
can occur in practice which either improve or degrade 
the system. It is good business practice to periodically 
audit the business system to assure that it is continuing 
to function as designed. Adjust the business system, dia- 
gram, procedure, and forms as necessary to meet the 
business policies in an effective and efficient manner. 

SUMMARY 

The FDA medical device good manufacturing practice regulations do 
not, in general, require more of your business than you should already 
be doing in order to maximize your long term profit. In most cases, 
what you do for your own business will satisfy the GMP requirements. 

Start with what you have. Change what you have to meet your own 
business needs. Adjust your business systems, if necessary, to meet 
the GMP. Document your business system requirements in policies, and 
your business system techniques in procedures. 

The result will be a business system which meets your own business 
needs first; and will also conform to GMP requirements. 
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PRODUCT LIABILITY IN EASTERN EUROPE 


John Mihalasky, Professor of Industrial Engineering* 
New Jersey Institute of Technology 
Newark, New Jersey. 07102 


INTRODUCTION 

To correctly understand the status of Product Liability (PL) in Eastern 
Europe it is first necessary to understand some economic and legal aspects of 
the Socialist countries of Eastern Europe. Then the status of PL will be pre- 
sented on a domestic level and separately on an international level. 

This paper does not attempt to pass on the effectiveness or quality of the 
systems, methods, and approaches described as operating in the Socialist Eastern 
European Countries. The presentation will be of facts and information as known 
to the author. 

Ownership of Business and Industry 

Industry and business is organized into enterprises, corporations and 
companies as in the U.S.A. These enterprises are then grouped into industries 
which are then grouped under ministries. Ownership is in the hands of "the 
people". Gross profits are distributed to such areas as capitalization, expan- 
sion, and new product lines, while the net profits (dividends) go to the general 
revenues for use in the economy. 

Manufacturing companies, research and development institutes, medical facili- 
ties, and insurance companies are all so state-owned. 

Citizen Benefits 


As is common with Socialist countries, the population is provided with an 
extensive social program. This program includes medical, dental, surgical, and 
hospitalization coverages. A form of what would be called Workman's Compensation 
is also available, along with coverage of lost wages for a great variety of reasons. 
These Eastern European Socialist countries also have a "full employment policy". 

Able workers are to be employed, or they can be charged with being "parasites" 
and, thus, subject to arrest. Therefore, the worker has to be offered a job where 
he can earn a living. 

Legal Environment 

The Eastern European nations (EEN) have various laws relating to industrial 
safety. These laws usually cover workplace safety, with some laws also dealing 
with the environment, both inside the enterprise and in the general outdoors. 

Federal Quality Control laws also exist. For example, Czechoslovakia (CSSR) has 
had a State Quality Control Act for at least ten years, and they are currently 
in the process of revising and updating this act. One of the expected results 
will be a model total Q.C. system. 

The USSR has been engaged in activities aimed at improving their legal 
assurance of product quality for the last ten years, and also expects to have 
a federal model for technical control departments in industrial enterprises, a 
model for quality inspectorates in industrial ministries, and for testing 
centers. This work is currently going on, headed by the State Committee for 
Standards of the USSR Council of Ministers. 


*Professor Mihalasky spent the 1976-1977 academic year as a Scientific Exchangee, 
of the U.S. National Academy of Sciences, attached to the Institute of Information 
Theory and Automation, of the Czechoslovak Academy of Science, Prague, 
Czechoslovakia . 
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The Socialist bloc also uses the technique of arbitration rather than going 
to the courts as is done in the U.S.A. to settle questions of warranty, poor 
quality, poor reliability, and other breeches of contract. 

Standards in the EEN have the weight of law, being mandatory. Non-conformance 
to standards is prosecuteable, resulting in material (economic), disciplinary 
and/or criminal actions against those responsible. At the present, the EEN are 
working on standardizing-harmonizing-standards between them. 

Standards are called out in contracts, or if not, the applicable standards 
pertaining to the product or service are in-force. A contract cannot be made for 
a quality level lower than the one specified in a state standard, unless the 
appropriate standard setting body gives prior approval, and an appropriate price 
reduction is made. When standards are violated, there are a specific set of rules 
governing applicable penalties for enterprises, management personnel, and workers. 
Criminal charges may be brought in cases of repeated and flagrant production and 
shipment of product that creates a danger to the consumer's health and safety, 
brings about losses for the enterprise, and/or losses for the economy as a whole. 

Product liability insurance as it is known in the U.S.A. is not offered to 
enterprises by insurance companies in the EEN, although liability insurance 
as a form is known and used, e.g., automobile liability insurance is required 
in EEN. 

The EEN also regulate the quality levels of their imports and exports. 

State inspectorates grade domestic product into classifications that permit 
some to be sold in domestic or international markets, e.g., Grades I and II; 
some to be sold only in domestic markets, e.g.. Grade III; and some to be sold 
only domestically, at discounts, as other than first-class product, e.g., 
product not meeting at least Grade III. 

Imported product is also inspected and tested for conformance to quality 
levels spelled out in contracts. In this activity, quality marks may be used 
to mitigate the need for such inspections and tests. 


THE STATUS OF P.L. ON THE DOMESTIC SCENE 

Complaints about poor quality have been on the increase in the EEN. The 
greatest percentage of the increase has come from industrial consumers that are 
demanding stricter adherence to standards, and an increase in the quality levels 
spelled out in these standards. 

Product Liability in the U.S.A. means liability by the manufacturer and 
other members in the distribution chain, for damages and injuries incurred by 
consumers and innocent third parties, when a product fails. The EEN do not 
have this concept of P.L. Their definition of P.L. is that of legal responsibility 
for poor quality in general - responsibility for product that does not conform 
to standards. 

Because of the social and legal conditions spelled out earlier in the paper, 
P.L. will - cannot - become the problem that it is in the U.S.A. In the case 
where a consumer is injured by a product, their social programs cover the costs 
of hospitalization, medical costs, and lost wages. As with most of Europe, and 
countries other than the U.S.A., suit for "pain and suffering" is not very 
common. The legal doctrine of strict liability in tort is not used, with reliance 
for assigning liability being on negligence and breech of warranty. 

The decision on where liability exists is made on (a) does the product 
conform to the standard, and/or (b) has the quality level that was warranted 
been violated. If liability is established, the fines are assessed as mentioned 
above, or in the case of an industrial consumer, the fine is determined by the 
process of arbitration. 

Product Safety, as distinct from P.L. , is receiving more attention in the 
EEN. As mentioned earlier, there are laws pertaining to the safety of products - 
safety being one of the quality characteristics of product. Several EEN were 
represented at the First European Seminar on P.L. hosted by the EOQC and the 
Italian Association for Quality Control, in Venice on 28-29 April 1977. As a 
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matter of fact, the delegate from the USSR, who presented one of the papers at 
the conference, expressed an interest in having the next conference in his 
country. The month following the Venice seminar, the first P.L. conference to 
be held in the EEN, took place in Prague, CSSR under the auspices of the 
National Committees for Quality and Reliability, of the Czechoslovak Scientific 
and Technical Society. 

They have recognized that the same techniques that have been used to attain 
high quality and reliability levels can also be used to attain high safety levels, 
e.g.. Design Review, FTA, FMEA, Data Collection and Retention. 

THE STATUS OF P.L. ON THE INTERNATIONAL SCENE 

On the international level, the P.L. situation begins to bear a greater 
resemblence to that found in Western Europe, Japan, and the U.S.A. This is 
because as the products of the EEN move into international markets, they are 
subject to the laws and conditions found in these markets. Product safety regu- 
lations and standards - whether voluntary or mandatory - have to be met. 

Industrial safety - OSHA type-laws also have to be contended with. 

The author's experience has been that product from the EEN has as far back 
as three - four years ago, been involved in P.L. litigation in the U.S.A. , and 
Eastern European companies in some cases have been unable to ship their product 
to the West because of the inability to get P.L. insurance - a prerequisite to 
entering the market. 

As laws and legal theories change in the West, the EEN will have to not only 
be aware of them but conform to them. Therefore, as their products for interna- 
tional trade will be changed to meet changing standards relating to product 
safety, their domestically consumed product will also reflect these changes. 

It would be uneconomical and inefficient to carry on a two-tiered product line. 


CONCLUSIONS 

Domestically, the emphasis in the EEN is developing in what in the U.S.A. 
would be called Product Safety .rather than P.L. However, on the international 
level, some of the P.L. problems that have been experienced in the U.S.A. are 
making their way over there. 

Their concept of P.L. is being developed within the efforts on legal 
problems of product quality control. To them, P. L. means responsibility for 
poor quality in general, not responsibility for damage to property and injury 
to person that arises from product failures. Their criterion of poor quality 
is basically that it did not conform to the mandatory standard. 

Adherence to standard, misuse or misapplication of product, and adherence 
to warranty terms are all acceptable defenses against claims of poor quality. 

Where liability for poor quality is established, fines are assessed 
according to pre-determined formulae spelled out in the laws, or determined 
through arbitration. This procedure applies to cases where poor quality 
causes bodily injury, property damage, and economic loss. As for the injured 
person(s), the social programs compensate them according to state law, and legal 
action’ is not resorted to. 
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CONTRIBUTION OF QC TO PRODUCT LIABILITY AVERSION 


Louis P. Geoffrion, CQE, 
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INTRODUCTION 


Fellow quality professionals, many speakers at meetings like this, start their 
presentations with a joke or two - Today I am not going to do that because I 
consider the current product liability situation facing American industry to be 
a very serious matter. 

This afternoon I plan to review some of the basic issues in product liability 
today and review some statistics from the most current source material. I will 
attempt to indicate to you the seriousness of the problem and it's possible 
impacts on industry in the U.S. and then outline for you some techniques which 
can assist you in developing quality control oriented strategies to avert or at 
least minimize product liability risks in your companies. 

First, let's review some basic definitions to insure that we are all hearing 
and understanding the same thing. 

Please bear with me if these definitions are second nature to you, perhaps 
some of us are new to these ideas. 

What is product liabili ty? 

There are many differing ideas on this subject which we hear so often BUT , 
basically, product liability is the simple concept that a provider of goods or 
services has an obligation to the recipient or consumer to insure that the 
consumer is not injured by their product and that _if there is an injury, the 
injured person will receive some compensation. 

Many people consider non- injury situations such as warranty claims, and 
other related problems to be a product liability issue but a warranty claim is 
really a problem of contract compliance to be precise and I do not intend to 
discuss this issue today. 


Why do we seem to have so many problems toda y? 

Fundamentally, because of changes in society which are reflected in judicial 
actions and in legislation. 

How many of us have heard the expression "caveat emptor" or "let the buyer 
beware"? 

Not too long ago, the phrase seemed to govern many product/consumer relations. 

In the last 6-10 years our society and for that matter much of the world 1 s 
society have repudiated that concept and today's product users have increased 
their expectations as to what they will receive when they purchase an item or 
service so that now the slogan might be better stated "caveat vendor". 

Why has this occurred? 

Well, a lot of people mutter labels such as "Ralph Nader", liberals, and 
other less complimentary phrases but suffice it to say that there are probably 
many causes which could be talked about all afternoon, let's just say that it 
has happened ! 
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Is there a crisis today? 


Well, President Ford established a project in mid - 1976 called the Interagenc) 
Task Force on Product Liability - pulling together many Federal agencies which has 
generated as Gov't bodies seem to do, a lot of reports. At the briefing meeting 
held at the Dept, of Commerce, in January 1977, chaired by the then Under Secretary 
of Commerce, Mr. Vetter, the conclusion was presented that NO crises existed in 
general in U.S. industry, though there were some problems. TD 

Now, admittedly, from the Washington viewpoint looking at many thousands of 
companies across the country there may not be a crisis but if you are a small 
manufacturer who suddenly can't obtain product liability insurance at any price 
you might have a different viewpoint. 

Nevertheless, some facts have been published which partially indicate the 
magnitude of the product liability problem today. 

A review of a number of trade association surveys of product liability 
experience has indicated that substantial increases in product liability insurance 
premiums have been experienced in the 1970' s. 

As depicted in fig. 1, in the period 1969 - 1974 average product liability 
insurance premium increase for manufacturers was 1547. while in the period 1974 - 
1976 the average increase was 3207.. (2) 



Fig, 1 Increase in product liability premium costs 

Many of these companies reporting premium increases had never experienced a 
single product liability claim and some were companies with a long tradition of 
safe, quality products. 

I am sure that we have all read of spectacular jumps in premium reimbursement 
of 4007. to as much as 19007. but this slide reflects data from the Task Force 
Industry Report and AMIA reports. 

Due to the high costs of insurance and also due to non-availability of 
insurance to some firms, many people are forced to operate w/o insurance, as 
shown in Fig. 2, a substantial number of small manufacturers operate in this 
manner. (3) 
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Fig. 2 U.S. Manufacturers without Product Liability Insurance 
- 1976 

The average amount of damages per firm in the survey claimed in new law suits 
rose from $476K in 1971 to $1.7 million in 1976. (4) 

Certainly, these numbers should make us all stop to think. 

Another significant statistic to consider is designed to make those of you 
sitting thinking, "I'm involved in industrial products, thank God I don't have the 
problems of consumer products QC"i 

Don't be too smug, you're not very safe - a recent insurance industry survey 
indicated that 71% of product liability claims over $100,000 paid in 1975 by the 
surveyed companies involved industrial products, not consumer products with an 
average cost/claim of $287,964. (5) 

This seems to be counter to popular opinion, why? 

The basic problem here is inadequacies, real or perceived in workman's 
compensation programs around the country. Don't forget, a worker accepting work- 
men's compensation treatments cannot sue his employer. 

If the injured worker receives a settlement from his employer's workmen's 
compensation insurance carrier which doesn't meet his or her expectations, their 
only recourse is to sue the manufacturer of the machine involved in the injury 
and this clearly has significant effect. 

An Example 


What many of us in this room would like to consider as rational common sense 
rarely seems to apply in these cases. Recently a QC inspector was injured, losing 
a portion of his index finger when he attempted to un-jam a conveyor drive chain. 
Prior to the accident this individual had been warned several times, by his 
supervisor, to call maintenance if there was any problem and any moderately 
intelligent person could be considered to know enough to stop a machine before 
poking at a drive chain. These concepts should certainly have protected the 
conveyor manufacturer from a lawsuit. Oh, by the way, there are a couple of 
other interesting facts here, including the age of the conveyor, 23 years , the 
fact that it was in the hands of a 3rd owner and that when built 23 years back it 
had been driven by a well guarded V-belt not an unguarded chain drive! 

Nevertheless the conveyor manufacturer still paid out $44,000 to the injured 
operator. 

Reasonable? Perhaps not, but it happened. 


What can be done? 


Now that I have set the scene, instead of attempting to resolve issues which 
will only be responsive to fundamental changes in society and the law, I think we 
should spend some time looking at what can be done to reduce the risks of product 
liability problems facing manufacturers today. 

You may say at this point, it's a simple task, "all you have to do is make the 
products to the specifications" and you won't have any product liability problems! 

Unfortunately, I'm afraid that won 1 1 work. 

But fortunately, from a QC practitioner point of view, I think that the 
present product liability crisis situation represents a unique opportunity and 
challenge to QC people. 

If the QC profession steps forward and responds in the appropriate manner, 
not only will the problems be reduced but the position and image of QC as a 
management group who are significant contributors to profits will be enhanced! 

This challenge can be met successfully and simply by remembering the funda- 
mental principle of Quality Assurance/Quality Control. That is, "to insure 
satisfaction of the customer's needs". 

Now, of course, we may need to do a few things differently, or with a 
slightly different emphasis to be sure of maximizing the effectiveness of our 
product liability efforts but ultimately, fulfilling this credo will assist our 
mission immensely. 

By the way, I consider those of us in the QC discipline to be among the most 
capable people in industry to accomplish this task because of our current respon- 
sibilities and activities in most of American industry. 

Some of the following suggestions, by the way, are going to sound like "how 
to establish a good QC program" but after all, that is my theme so bear with me, 
and think as I talk how you could expand your existing programs to include 
product liability considerations. 

To begin - first step, and a very important step, is the promulgation by top 
management of a clear and unequivocal policy which basically states that the 
company wants to provide it's customers with safe products. 

As we all know from experience, without this kind of policy statement, and 
committment and support, QC's efforts will be stymied to a great extent. 

Next, we have to establish a mechanism to accomplish this policy; and since 
I'm addressing my remarks to QC people from large and small companies, please 
accept my statements as general recommendations which may be implemented with a 
slight modification or change in emphasis to an already existing QC system. 

The most effective mechanism to implement this policy is a mult-disciplinary 
committee charged with product safety and liability exposure considerations. 

My preferred organization concept for this Product Safety Advisory committee 
is as shown in Fig, 3. 

This committee should exist at the top level of the organization and will 
basically establish detail policy and procedures for subordinate levels in the 
organization. 

If you're a small company QC Chief this may be all the organization there is 
need for, if so action will be required as well as policy establishment. 

For the larger firm, suitable committees of similar structures should be 
established down to the necessary levels in the firm usually where actual design 
or development activities are carried out. 
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Fig. 3 Product Safety Advisory Committee 


Regular meetings with carefully documented notes of actions, decisions, and 
recommendations should become an important part of the QC paperwork file at each 
committee level, with upward and downward communications mobility. 

I think that having QC be in charge of this type of committee is a logical 
extension of the modern concepts of QC's role in industry reporting to top 
management for maximum organizational effectiveness. 

What else should our basic policy state? 

Additional elements of basic policy - Your firm's activities should include 
the incorporation of "state-of-the-art" safety engineering principles in all 
phases of design and manufacture. 

Now, I'm assuming, that this is a new way of looking at an existing organizat- 
ions' activities and methods. Any organization has at any one time both new 
products and existing products. It is one thing to set up a system to handle new 
products or, designs, but what about those products which have been around for 
awhile? 

Actually, this can be handled quite simply: 

Starting from establishment of your new policy, throughout R&D and product 
design and manufacture, state-of-the-art safety engineering principles will be 
incorporated in all new product development. In addition, an ongoing review of 
existing product designs will be established and maintained to provide for 
appropriate product modifications relevent to the current state-of-the-art. 

Clearly, since no one has infinite resources to handle this review process, 
some system of prioritization is needed to decide which of the existing designs 
will be looked at first. 

I would like to suggest that this prioritization be determined by assessment 
of the risk exposure of the various product's in your company's catalog and that 
this assessment: should be a function of the Product Safety Advisory Committee. 

The amount and depth of design review for product safety considerations will vary 
enormously between different companies so I will offer some common, general 
principles - 

1. Review the design according to generally accepted engineering/ 
scientific principles from a safety consideration 

2. Review your product's design against standards applicable in 
the industry 

3. Review your product's design in comparison with competitive products 

4. Possible risk exposure inherent in product 

5. Risk potential from product misuse 

6. Document all decisions carefully and completely 

Make sure that all new product designs are tested to validate the design 
safety and quality objectives and that existing products have adequate records of 
product qualification. 


Insure that adequate QC, of all aspects of product manufacture is in existence 
to insure the ongoing accomplishment of the design safety and quality objectives,. 
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Insure that adequate records of operations, inspections and tests are available 
to document the accomplishment of the ongoing QC effort even in a courtroom. 

Insure that all operating instructions, product labels, product bulletins, 
catalogs and advertising consider the user's safety and the producer's exposure to 
product liability risk. 

Plan for the worst 


There is another facet of my program which must be looked at now which may 
never be needed but certainly must be part of your planning. 

What do you do when all the actions listed earlier have been done and you still 
get a letter from a lawyer, or you become aware of an "incident" involving one of 
your products? 

Depending on the complexity of your organization and the degree of risk 
exposure that your products bear, your response may vary. But the need for a 
planned respone pattern is definite. 

Basically, each company should have a system designed to obtain all pertinent 
facts concerning any incident which causes injury, death, or damage to property. 
Obtaining possession of the product involved in an incident is highly desirable 
also but if this happens a system of careful control of the evidence must be 
maintained to make sure that the information is of use in court defense. 

All of this information must be communicated rapidly and completely to 
legal and insurance functions and this can be most effectively handled through 
the committee structure though the timeliness of the system must be kept in mind. 

For those of you interested in the detail of "what to do after the incident", 

I would recommend your obtaining a copy of the excellent paper presented at the 
PLP77E by Mr. James Gardner. (6) 

Conclusion 


In conclusion, I would like to state firmly that the situation in this country 
concerning product liability exposure is going to get worse before it gets better 
at least from a manufacturer's viewpoint and that the "getting better" will quite 
likely be years from now. 

The argument that, ultimately, society will be served through the exercise of 
the free market system is only applicable in the long term and unfortunately most 
of us have to operate on a day to day basis in our jobs. 

To insure the short and long term capability of your company to withstand the 
problems of product liability risks and attendant insurance problems and costs a 
system and plan are needed. 

I maintain that such a system, following the outline I have presented, can 
best be developed by an aggressive and capable group since these concepts are 
extensions of well accepted QC principles. 

Ultimately, a well run Product Liability Risk Aversion program will have a 
positive effect on profits and, after all, that is the goal of all management 
activities • 


LCS 354:10:400 
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1 NTRODUCT I ON 

There have been a significant number of words written about safety, reliability 
and fault tree analysis. After many of those references have been read and studied, 
the thought occurs that perhaps another set of words, organized though they may be, is 
not needed. Yet, the human inclination to restructure the previous work of others, 
with the hope that a small contribution can be made in the process, is difficult to 
subdue. 

This paper wi II define, compare, and discuss safety and reliability. It will also 
present the rudiments of fault tree analysis, which can be used to analyze the safety 
and/or reliability of a device, product, system, process, or service; usually "product” 
will hereafter be used to mean any or all of these terms. 


SAFETY - RELIABILITY 

It is generally agreed that a safe product is not necessarily a reliable product; 
the converse is also true. The significance of that statement, however, is not always 
so generally appreciated by managers, designers, and others who are responsible for 
the products appearing in the marketplace. The well meaning people that put out a 
commercial paper cutter that "reliably" cuts arms as well as paper did not appreciate 
that fact, nor did the people who shipped some very safe electrical appliances, with- 
out cords. The latter problem is merely a nuisance; the former is deadly. 

There are many situations in which reliability does imply safety, at least to some 
degree. If the light screen enclosing a press reliably detects an object which inter- 
sects it, at least one element of the system is contributing to the safety of the press 
Note that it has not been said that in this case reliability is synonymous with safety; 
it only assured that one part of the press control operated in a safe mode. 

Graphically, the partial intersection of reliability and safety can be illustrated 
by a Venn diagram of Figure I. 



The double cross-hatched area represents those situations where the reliable 
functioning of a product also implies a safe mode of operation. Of course, it is 
never possible to obtain total intersection of the two domains for several reasons: 
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all hazards can never be known; the domains will probably not be the same size; the 
time and funds are not available to do so. 

Before continuing, it would be well to define a few concepts and terms. Human 
safety seems to be the dominant interest and concern of most people, although equipment 
and economic safety should not be forgotten. It is product liability litigation, with 
its consequent economic implications, which has placed the safety of people in the 
foremost position. Government agencies such as the Consumer Product Safety Commission, 
have also accentuated personnel safety above all others. It is reasonable that the 
discussion of safety primarily concerns the well being of people. That which is said 
about the safety of people would generally apply to the safety of equipment. 

Safety is defined as a judgement of the acceptability of risk. Risk is a measure 
of the probability and severity of harm. It can quite succinctly be said, then, that 
"A thing is safe if itsrisksare judged to be acceptable." CG All that is succinct is 
not simple, alas. The risk must be measured. This can be accomop I i shed , but it re- 
quires the accumulation of data, and is probabilistic in nature. The measurement of 
risk is objective. The second ingredient of safety, judging the risk acceptability, is 
quite subjective, however. The judgement or evaluation process is a function of per- 
sonal and cultural background, technical sophistication, familiarity, prior exposure 
and many other human ingredients. Lowrance [j J devotes a chapter to each of these two 
facets of safety. For additional background on risk perception, the reader should ex- 
amine several works by Chauncey Starr C2D> C3ll. 

Reliability is defined in terms of operating function, time and environment. 

Among the many similar definitions available, NASA defines reliability as "the prob- 
ability of a device performing adequately for the period of time intended under the 
operating conditions encountered." The reliability of an operating product can be 
objectively measured with data accumulated over the operating period. However, pre- 
diction of the reliability of an identical or similar product is a subjective concept. 
This is a subjective step because of the belief that the data from the operating pro- 
duct applies to the similar product. Thus, although reliability can be objecti ve ly 
measured, it is probabilistic in nature, and its application for prediction is a some- 
what subjective operation. 

The probabilistic nature of both reliability and risk makes it impossible to pre- 
dict the outcome of any discrete event. Only the average outcome of many events can 
be predicted. Even this is predicated on the subjective notion that the average 
(probable) outcome of a set of previous events is representat i ve of a set of future 
events. Safety, in fact, has two subjective aspects to it, i.e., that which is 
associated with risk as above, and that due to the judgement of the acceptability of 
the risk. 

Because of the probabilistic nature of reliability and safety, it is important 
to optimize their probabilities, consistent with reasonable cost-benefit decisions. It 
is vital to know, as well as practicable, the values of the various event probabilities 
and the nature of the sensitive segments of the product; fault tree analysis does this 
well. 


FAULT TREE ANALYSIS 

A fault tree is a graphical record of an analyst’s logical and systematic answers 
to the question, "In what ways could a given undesired event happen?" It is a syste- 
matic, deductive analytical method to identify those combinations of events which will 
result in the undesired top event. For example, an undesired top event might be 
"Overhead Projector Fails to Light." With the top event defined, the formalized pro- 
cess using event symbols and logic gates to link basic events to the top event can be- 
gin. Among the requirements for a successful fault tree are a thorough knowledge of 
the product, a logical thought process and a bit of creative foresight. 

After a fault tree has been structured, it can be evaluated either qualitatively 
or quantitatively or both. The end product of the effort should be a listing of the 
critical paths which lead from basic events to the top event. This identifies the 
minimal sets of components that can cause failure of the product (top event) if the 
components fail. A qualitative analysis determines the critical path, or min cuts. 

A quantitative analysis requires data in the form of probabilities. Although 
component failure rate data is available, it is not likely to be read i I y available in 



most useful form for all components of interest. Some may have to be developed; some 
may have to be ’’adapted" from related component data. The most difficult "component" 
to quantify in probalistic terms is human behavior; it is most likely to introduce 
error in any quantitative analysis. Because of these difficulties, there are those 
who take a dim view of the value of a quantitative analysis based on such "questionable" 
data. They refer to such efforts as worship of numerology. There are others, however, 
who would prefer to have the quantitative analysis based on the best avai lable data 
while taking a realistic view of the probable error it contains. They feel that it is 
better to have numerical results of possible substantial error, and use them carefully, 
than to have no quantitative results at all. 

The request was that this paper should explain the construction and use of fault 
trees in safety and reliability analyses. Consequently, the reader who is well ac- 
quainted with fault trees will find little that is novel. The reader who knows what 
a digraph is, should go on to the next paper. For the rest who are not well acquain- 
ted with tree structures, there may be something to be learned. For those who wish to 
study the subject beyond this paper, Brandell M offers a short, easily read treat- 
ment. Another fine introduction appears in an American Gas Association publication [bD. 
Buys has written a generalized guide for structuring trees E6D. Two, among many, works 
by Lambert [7], [8]] provide an excellent and comprehensive treatment; they ore parti- 
cularly recommended. A SIAM conference proceedings [9] contains a large number of 
more sophisticated, mathematically oriented papers. Brown's book [JOD demonstrates 
the use of fault tree analysis for cost-benefit analysis. Lapp and Powers [j 0 pre- 
sent an algorithm for the synthesis of fault trees. The above suggested references 
ore not exhaustive by any means, but will lead the reader to most of the significant 
literature on fault trees. 


EVENT SYMBOLS 

An "event" is the change of a component condition from one state to another. If 
the change of state occurs according to specification, the event is said to be normal. 

If the change is to an undesired state, or if the state of the component does not 
change when it should, the event is said to be abnormal; it is a fault event. It 
should be noted that every component failure is not a fault event. A fault is a com- 
ponent state that may somehow lead to the undesired (top) event. A component failure 
that does not enable the undesircd event to occur is not a fault. 

The basic event symbols which depict normal and fault events include the rectangle, 
circle, diamond, house, oval and triangles, as shown in Figure 2. 
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FIGURE 2 EVENT SYMBOLS 


The rectangle event is called a gate event because it is the output of a logi 
gate. It depends on the contributory events acting through the logic gate, and on 
the type of logic gate to which it is connected. 




The circle event is a basic event that requires no further development. It is 
always an input to a logic gate, depending on no other event. It is a characteristic 
failure of a produt when used within its design envelope. It is considered a primary 
failure (primal). It is the result of a random failure. 

The diamond event is one which is not further developed because it is not signi- 
ficant, there is not enough data, time or money, or when further development would 
duplicate information already known. It is considered a basic event. 

The oval defines a conditional input to an inhibit gate. It specifies a condition 
that must exist to enable an input fault to cause an output from the gate. The condi- 
tion may be normal or may be a failure mode. 

A house event is one that is normal for the product and is expected to occur. It 
is not a fault or failure, per se. 

The triangles are transfer symbols. Each pair will have a unique alpha-numeric 
symbol. The triangle with the line at the side specifies that the entire segment of 
the tree below the point at which the line is attached will be transferred to another 
part of the tree. The triangle with the line at the apex identifies the point on the 
tree to which a segment will be transferred. 

The statement, "Projector fails to light," is a rectangular event. It could be 
an output statement from a logic gate at any level in a fault tree. The event state- 
ment appears in the rectangle as in Figure 3. Note that it consists of, a subject, 
verb and descriptor; see Figure 4. 
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In this case, the descriptor is the word "light." The mode of failure is clearly in- 
dicated. Similarly, the statement "Switch Fails Open" indicates that the state of the 
switch which may contribute to the occurrance of the top event is the open state. This 
format for event statements is especially important for quantitative analysis since the 
data for the probability that the "switch fails open" is different from that for the 
probability that the "switch fails" (in any mode). For consistent, logical structuring, 
the format of subject, verb, and descriptor should be used whether the fault tree will 
be used for qualitative or quantitative analysis. 


LOGIC GATES 

There are basically only two logic symbols or logic gates used in the construction 
of a fault tree, the AND gate and the OR gate. Sometimes the (•) and (+) mat+’remat i ca I 
symbols are added to the gates to emphasize which is which. The dot implies logical 
intersection in set theory; the plus sign denotes logical union. See Figure 5. 



FIGURE 5 EXAMPLE OF AND and OR GATES 


















AND and OR gates are the primary logic gates used in fault tree or other tree 
analyses. Any other logic gates are adaptations of the two basic gates. As an 
analyst becomes more sophisticated, he will use and "invent" a number of special gates. 
Among special logic gates are the inhibit gate, priority AND gate, excluxive OR gate, 
summation gate and other AND gates with unilateral or mutual dependence.' Not all of 
the above will be discussed here. 

The inhibit gate is a single input AND gate with an enabling condition which must 
exist for the possibi I ity of an output to occur. The condition may be. either normal 
or abnormal to the product. The probability of the output is P(t)xC. P(t) is the 
probability of the input; C is the probability that the conditional event occurs given 
that the input has occurred. An example of the inhibit gate is given in Figure 8. 



FIGURE 8 INHIBIT GATE EXAMPLE 


The priority AND gate is an example of a gate for which the events have unilateral 
dependance. The input events must all occur and they must occur in a particular se- 
quence in order for an output to occur. The unilateral dependance of a priority AND 
gate may be specified by a condition statement such as "event I before event 2," or by 
the format of Figure 9. This example implies that after the original lamp burns out 
an attempt is made to switch the spare lamp into the socket, but the mechanism fails. 
The first lamp must fail before the switch attempt is made. 
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FIGURE 9 


AND GATE WITH UNILATERAL DEPENDENCE 


The exclusive OR gate will have an output if any one of the input events occurs, 
but will have no output if two or more input events occur. The summation gate requires 
that an acceptable combination of input events be present to produce on output. Inputs 
can be present in varying proportions although a minimum sum of inputs is necessary to 
obtain an output. 


FAULT TREE CONSTRUCTION PROCESS 


There is no mandatory or consensus standard method for going about the job of 
structuring a fault tree. However, Lambert C7j| has suggested a rather well thought 




out process; the nine rules are reproduced below: 

Rule I : State the fault event as a fault, including the description and timing of a 
fault condition at some particular time. Include (a) What the fault state of that 
system or component is; (b) When that system or component is in the fault state. Test 
the fault event by asking (c) Is it a fault?; (d) Is the what-and-when portion' included 
in the fault statement? 

Rule 2 : There are two basic types of fault statements, state-of-system and state-of- 
component. To continue the tree (a) If the fault statement is a state-of-system state- 
ment, use Rule 3. (b) If the fault statement is a state-of-component statement, use 

Rule 4. 

Rule 3 : A state-of-system fault may use an AND, OR, or inhibit gate or no gate at all. 
To determine which gate to use, the faults must be the (a) Minimum necessary and suf- 
ficient fault events. (b) Immediate fault events. To continue, state the fault events 
input into the appropriate gate. 

Rule 4 : A state-of-component fault always uses an OR gate. To continue, look for the 
primary , secondary, and command failure fault events. Then state those fault events. 

(a) Primary failure is failure of that component within the design envelope or environ- 
ment. (b) Secondary failures are failures of that component due to excessive environ- 
ments exceeding the design environment, (c) Command faults are inadvertent operation 
of the component because of a failure of a control element. 

Rule 5 : No gate-to-gate relationships. 

Rule 6 : Expect no miracles; those things that would normally occur as the result of a 
fault wi II occur, and only those things. Also, norma! system operation may be expected 
to occur when faults occur. 

Rule 7 : In an OR gate, the input does not cause output. If any input exists, the out- 
put exists. Fault events under the gate may be restatement of the output events. 

Rule 8 : An AND gate defines a causal relationship. If the input events coexist, the 
output is produced. 

Rule 9 : An inhibit gate describes a causal relationship between one fault and another, 
but the indicated condition must be present. The fault is the direct and sole cause of 
the output when that specified condition is present. Inhibit conditions may be faults 
or situations, which is why AND and inhibit gates differ. 

An additional rule is to proceed in small steps; be verbose. 

Basically the above procedure involves two questions: (I) Is the event a state- 
of component or state-of-system fault? (2) What events are necessary and sufficient to 
result in the event in question (I)? 

A state-of-component fault exists if failure of the component under investigation 
can itself be the fault. An OR gate is always used below a state-of-component fault 
event. The input events to the OR gate are the following three: 

1. Primary failure - the component simply reaches the end of its life while being 
used within its design envelope. 

2. Secondary failure - premature component failure occurs due to excess stress 
associated with its being used outside the design envelope. 

3. Command fault - an unprogrammed operation of the component due to human or in- 
animate control mechanism failure, i.e., the right thing is done at the wrong 
time. 

Figure 7 is an example of the format used for a state-of-component fault. 

A state-of-system fault exists if a simple component failure cannot be the fault 
event. Below a state-of-system fault event an OR, AND, or inhibit gate may be used; 
occasionally no gate is necessary. The immediate minimum necessary and sufficient 
fault events for the state-of-system fault to occur will dictate which gate to use. 
Figure 6 is an example of a state-of-system fault, since the failure of the first 
lamp alone could not cause the output event. The spare lamp also had to be defective 
to cause the event ’’Lamp i n socket is open.” 

To develop the fault tree beyond the secondary failure and command fault of the 
state-of-component fault, it is important to recognize their nature. Command faults 
are always state-of-system faults. The development of a secondary failure requires 
the use of an inhibit gate C7j. 



A secondary failure of a component results from excess operational or environmen- 
tal stress. An operational stress would be an out-of-tolerance condition such as 
higher-than-speci f led current in a resistor. An environmental stress would be a 
detrimental energy input such as the heat resulting from the excess current in the 
resistor. Hammer [123 has a checklist of hazard sources. He also shows that the de- 
velopment of a secondary failure is detai led, requ i ring thorough product knowledge. 

The operation of a switch at the wrong time is a command fault. The failure of 
the switch itself could not cause the fault. Hence, it is not a state-of-component 
fault; it is a state-of-system fault. To develop a command fault, the possible reasons 
for the fault must be found. For example, a ma I funct ion i ng transducer could signal a 
human or control mechanism to take unnecessary, improper emergency action. 

AN example: 

Although the fault tree analyses of very complex problems have received widespread 
attention, the application of the method to relatively simple problems has merit be- 
cause of the logical thought process and the graphical record which results. Examina- 
tion of how a projector might fail during a lecture does not compare to a study of how 
safe a nuclear reactor is; however, such an exercise does give insight to the method. 

A narrative of how a product operates should be written prior to actual fault tree 
construction. Because of space constraints here, that step will be omitted. Secondary 
failures are listed separately for possible further analysis. The sample fault tree 
shown in Figure 10 is limited by the space availabe in this paper; it could be further 
deve I oped. 

CONCLUSION 

The concepts of safety and reliability have been defined and compared. It has 
been pointed out that the achievement of either can be improved by use of fault tree 
analysis. Some rudiments of fault tree analysis have been presented, with direction 
for significant sources of further information. An example fault tree for a simple 
event has been partially developed. 
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STANDARDS AND SAMPLING PLANS FOR PRODUCT SAFETY 


John A. Clements, Director 
Corporate Product Evaluation 
The Gillette Company 
Boston, Massachusetts 02106 


Testing consumer products for safety raises many interesting and important issues 
in the areas of: 

1) Human and engineering test methods, 

2) Safety standards, and 

3) Sampling plans used for testing conformance to standards. 

This paper seeks to provide some perspective on consumer and industry viewpoints 
on product safety and the setting of safety standards. It then goes on to review 
sampling methods which appear useful in testing conformance to safety standards. 

DISCUSSION OF CONSUMER AND INDUSTRY VIEWPOINTS ON PRODUCT SAFETY 

At first sight, it would seem that the consumer at large has two somewhat con- 
flicting requirements from today's products. First, he requires a product which is 
easy to use, and freely available at a reasonable price, but on the other hand, he 
wants the product to be completely safe. Likewise, industry is subjected to opposing 
pressures. First, a competitive market requires that manufacturing costs be kept to 
a minimum, but responsibility to the consumer, maintenance of market share, and confor- 
mance to the law require the highest degree of built-in safety possible. 

In order to find a common ground on which these conflicting positions can meet 
for the determination of safety standards and conformance thereto, it is necessary for 
industry and the consumer to carefully examine their objectives and requirements for 
product safety. 

First, industry must have as its primary objective good initial design quality, 
compatible with existing technology, which addresses all possible causes of injury in 
both use and reasonable misuse, and all known product environments. 

Second, industry must be devoted to good manufacturing codes which minimize prod- 
uct defects and ensure the detection of unacceptable defect levels prior to product 
being shipped. 

Third, industry must ensure that its labelling, packaging, warnings, and consumer 
education fully informs the consumer of all risks inherent in product use or reason- 
able misuse. 

The consumer for his part should: 

First: Be willing to exercise care and intelligence in the use and handling of 

the product, including adherence to appropriate use, maintenance and 
cautionary instructions provided by the manufacturer. 

Second: Be willing to accept that there is no such thing as absolute safety, and 

that levels of safety will vary with the state of technology for differ- 
ent products as well as consumer requirements for performance. 

Third: Be aware that there is always a finite (if small) probability of sus- 

taining an injury when using almost any product. 

If the above points are kept in mind, a rational approach to standards setting is 
possible, and acceptable sampling plans for conformance can be arrived at. 

PRODUCT SAFETY STANDARD SETTING 

The setting of a performance standard for consumer product safety requires three 
basic steps: 
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1) Determining product performance characteristics which are regarded as safety 
related. 

2) Defining precise methods of measuring the chosen characteristics on units of 
products . 

3) Specifying the range of values of these measurements (i.e., limits) which 
are regarded as satisfactory from a safety point of view. 

A fourth step which has been the subject of much discussion and controversy over 
the past few years is: 

4) Development of methods for assuring compliance to the standard. 

A paper by the National Bureau of Standards ^ discusses five methods of ensuring 
compliance, viz: 

1) Processing of complaints of injury and defective products. 

2) Market place sampling of products. 

3) Voluntary in-plant sampling or other Quality Control Assurance by the 
producer . 

4) Mandatory in-plant sampling prescribed by the regulator. 

5) Prototype testing prior to production. 

The last four of these methods require testing and therefore sampling of the 
product. 

CHOICE OF SAFETY LIMITS 

The following discussion lists the main issues involved in the initial setting of 
safety limits in the standard, insofar as they are relevant to the specification of 
sampling schemes. Some sampling plans and schemes are then described which can help 
to overcome some of the problems encountered. 

The main issues which cause difficulty in the actual setting of safety limits, 
and methods of sampling to ensure conformance, are as follows: 

1) The actual test methods and environmental conditions under which the tests 
are carried out cannot duplicate the wide range of real product use (or mis- 
use) conditions and environments. They are only approximations to real 
life . 

2) There is usually no unique point on the scale of measurement for a particu- 
lar characteristic which defines the line between safe and unsafe. Rather, 
there is more likely to be a gradual gradation from safe to unsafe along the 
scale of measurement. 

3) The relationship between the scale of measurement and level of safety (or 
probability of accident) is not directly measurable, and is rarely, if ever, 
known with any exactitude. In other words, although we would like to be 
able to plot the curve in Figure 1, we cannot do so. 



of Safety Parameter 
FIGURE 1 

4) The above lack of knowledge is further complicated by the lack of precisely 
measurable information on the probability and severity of injuries arising 
from given accidents, and the accident scenario. 

5) Optimum product performance is not usually at the point of minimum probabil- 
ity of accident, and optimum product performance is not the same for all 


consumers. There is often an average desirable product performance P, and 
a p max an <3 Pminf thus: 



Decreasing Safety 


Increasing Performance 
FIGURE 2 

6) Minimum product cost is not usually at the point of minimum probability of 
accidents, thus: 



Decreasing Safety 


Decreasing Costs 


FIGURE 3 


In addition to the variability of consumer requirements for performance, 
there is variability of the product being manufactured. This manifests it- 
self in two ways: first, as a distribution of values about some chosen tar 
get value, and second, the possible occurrence of mavericks, as shown in 
Figure 4. 



FIGURE 4 


8) The amount of variability may be further increased during the life of the 
product, due to ageing or storage effects. 

Bearing the above considerations in mind, it will be apparent that the determina 
tion of the safety limits is very much a matter of judgement, and will usually be a 
balance between performance requirements, technological and manufacturing capabili- 
ties, and attainment of an acceptable safety level. 



It is usual to choose a limit (L) so that there is only a small risk of accident 
below it, and at most only a small percentage of product made above it. 



FIGURE 5 

If this proves impossible, then either the target value, P, has to be moved down (at 
the expense of performance) , or production variability has to be reduced, to the point 
where a consumer acceptable limit is obtained. 

So long as each specific limit in a safety standard is set with these considera- 
tions in mind, consumers and regulatory agencies must recognize that some units of 
mass produced product which exceeded the limit will reach the market, but that these 
units do not represent a sudden jump from absolute safety to substantial hazard . The 
producer for his part must ensure that the risk of accident is kept to an absolute 
minimum compatible with: 

1) Consumer requirements for performance, which will determine the target or 
average value of the product distribution. 

2) The state of technology underlying design, which will determine both the 
location and spread of the distribution. 

3) The capability of manufacturing processes and materials in avoiding defec- 
tives which fall outride the distribution. 

SAMPLING PLANS FOR USE WITH SAFETY STANDARDS 

There has been much discussion in recent years concerning the part that sampling 
plans should or should not play in safety standards. This audience is probably famil- 
iar with many of the diverse opinions which have been aired on this topic. Some in- 
sist that nothing less than 100% cortpliance is satisfactory, while others would settle 
for large sample/zero defect sampling plans. There are also those who argue for more 
practicable small samples and allowable defects. Whether or not sampling plans should 
be included in safety standards is also a debatable issue, but that it is necessary to 
use well thought out sampling plans for checking conformance to standards is undeni- 
able. I am going to discuss three types of sampling plans which, if used separately 
or in combination, would contribute to a balanced compromise between the above men- 
tioned extreme points of view, and show how a judicious use of available sampling 
technology can be brought to bear on an urgent technical and social problem. 

The three types of plans I will discuss are: 

1) Three Class Attribute Plans. 

2) Normal/Tightened/Reduced Plans. 

3) Dodge & Romig type AOQL Plans. 

THREE CLASS ATTRIBUTE SAMPLING PLANS FOR SAFETY STANDARDS (2) 

The main advantage which 3-class attribute plans have in sampling for safety is 
the recognition of a "marginal zone" between "good" and "bad" product. In sampling 
for safety, this marginal zone can be considered as the "safety margin" between "rela- 
tively safe" and "relatively unsafe" product. A given proportion of product can then 
be tolerated within this "safety margin" zone. This proportion can be kept below 
agreed levels by normal Quality Control methods and compliance can be judged using re- 
latively small sample sizes with non-zero acceptance numbers. In addition, it is 
accepted that any product in the "relatively unsafe" zone is unacceptable, and no 
defects of this kind are allowed in the sample. 



Di agramma ti cal ly : 


Targe t 



Relatively Safe 
(good) 


Safety Margin 
(marginal) 


Relatively Unsafe 
(bad) 


FIGURE 6 

In a sense, the safety limit becomes a zone rather than a hard and fast line of demar- 
cation, but the scheme still retains a harsh zero defect control on product that is 
outside this zone. Thus, these plans provide a nice combination of control at the 
margin and protection against extremes. 

It is possible to visualize cases where high quality levels could be maintained 
using large sample sizes, and small (non-zero) acceptance numbers for bad product, 
but this paper will deal only with schemes which allow zero defects in the bad zone. 

Consider as an example the following plan. 

Sample size =10 

Allowable marginals = 2 

Allowable bad = 0 

The set of operating characteristic curves for this scheme are given in Figure 7. 
The curves show how the probability of acceptance varies jointly with the percentages 
of marginal and bad product in the batch. With 8.5% marginal and zero bad, the prob- 
ability of acceptance is 0.95. However, the plan reacts quickly to fail batches con- 
taining even low proportions of bad material, but passes most batches with lowish 
quantities of marginal product. 

Thus, assuming a producer is controlling the mean and spread of his distribution, 
a shift of either in the wrong direction will be detected by increasing quantities of 
marginal product, and this would provide a warning to both the producer and the regu- 
latory agency without incurring the penalty of failing an inordinate number of 
batches. 

On the other hand, a sudden occurrence of gross defects (mavericks) which were 
not part of the general distribution would be detected quickly. 

Other plans with sample sizes up to 150 are given in Reference 2, and these show 
that by a judicious choice of acceptance numbers for marginals, the operating charac- 
teristics can be moved to the left or right and steepened. Of course, as the sample 
size increases, the plans become increasingly harsher on bad product. 

NORMAL/TIGHTENED PLANS 

The three class attribute plans described above can be further modified by using 
an appropriate normal/tightened strategy. The strategy adopted will depend on how 
critical the characteristic is to product safety, as well as the economics of sampling 
and testing. For example, if a particular limit is considered highly critical to 
safety, but testing is destructive and expensive, a tightened/normal/tightened system 
as follows might be considered: 

Acceptance # 

n Marginals Switching Instructions 


Tightened 60 12 Switch to normal after 5 acceptable batches . 

Normal 10 2 Switch to tightened if 2 out of 5 consecutive 

batches rejected. 

The operating characteristic is shown in Figure 8. 

This plan is much harsher to bad product than the normal plan above, and also 
more discriminating between low and high percentages of marginal product. 



Of course, it is only possible to use such normal/ tightened plans where production 
is continuous, and where the producer is doing the actual sampling and testing on site. 

AVERAGE OUTGOING QUALITY LIMIT (AOQL) PLANS 

As the main object of the above plans is to ensure that only a minimum proportion 
of product in tie safety margin zone reaches the market, there is merit in using an ap- 
proach similar to that contained in Dodge & Romig tables (3) in which an AOQL for mar- 
ginals is agreed upon and the level of sampling is determined by the process average 
of production and lot size. Such schemes recognize that different production process- 
es will produce different levels of percent defective, and they provide an economic 
stimulus to the good producer by requiring lower sample sizes when the process average 
is lower. It is usually felt that 100% screening of rejected lots is a prerequisite 
to the use of the AOQL concept. However, as Dodge & Romig have pointed out, product 
having a percent defective moderately higher than say 1.5 times the AOQL will suffer 
sufficient rejections to force the producer to undertake corrective action to improve 
his quality. Thus, together with potential economics in sampling, there is a double 
pressure on the producer to get his quality well above that called for by the AOQL. 

There is no reason why Dodge & Romig and 3 class attribute schemes should not be 
combined. In such schemes the allowable number of units in the bad zone would still 
be zero, and the sample size and acceptance number for marginal product would be 
determined from Dodge & Romig tables. The operating characteristics given in the 
tables would then apply to the zero percent bad condition. 

For example, if up to 15% "marginal" product was considered acceptable, and a 
producer had a good history of producing virtually no bad products, the following plan 
adapted from Dodge & Romig Sampling Inspection Tables could be used. 


Batch Size = 50,000 - 100,000 


n 


Cl 

£2 

Marginal 
Process 
Ave rage % 

Bad 

Process 
Average % 

Marginal 

AOQL 

% 

Probability of 
Accepting Process 
Average Batches 

P ( ♦ 15) 

14 


2 

0 

0-0.2 

0 

10 

.99 

.65 

19 


3 

0 

0.2-2 

0 

10 

.99 

.69 

44 


7 

0 

2-4 

0 

10 

.99 

.66 

80 


12 

0 

4-6 

0 

10 

.99 

.64 

155 


22 

0 

6-8 

0 

10 

.99 

.50 

260 


35 

0 

8-10 

0 

10 

.99-. 96 

.29 


n 

= 

Sample 

size 






Cl 

= 

Acceptance number 

for marginal product 




C2 

= 

Acceptance number 

for bad product 




p( 

15) 

= 

Probability of accepting batches 

containing 15% marginal product. 



This 

plan has 

the following advantages. 

it: 




1) 


Requires 

low sample 

sizes when process average 

for marginals is low. 



2) 


Provides 

a very high probability of acceptance 

when process average 

for 


marginals is maintained. 

3) Forces action for improvement by rejecting more than 1 out of 3 batches 
when marginal product exceeds 15%. 


4) Produces an AOQL of 10% marginals if rejected batches are screened. 

5) Results in minimal total inspection when rejected batches are screened. 

6) In general, encourages producers to keep distributions of important safety 

parameters well within the safety limits. 

7) Gives more stringent protection against accepting bad product, as process 

average for marginals worsens. 
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VALUE OF AHAM STANDARDS AND CERTIFICATION PROGRAMS 


Herbert Phillips 
Vice President Engineering 
General Manager Certification 
Association of Home Appliance Manufacturers 
Chicago, Illinois 60606 


In order to set the stage for this paper, I will very briefly describe AHAM’s 
standards development activities. 


First, AHAM develops product performance standards for home appliance products. 

A few of these are simply standard methods for measuring performance; others, in addi 
tion to being test methods for measuring performance, prescribe some realistic recom- 
mended levels of performance. See AHAM "Index of Standards", Figure 1. 


FIGURE 1 


AHAM PUBLISHED STANDARDS 


DH-l — Dehumidifiers, Sell-Contained, Electrically 
Operated. Mechanically Refrigerated (American 
National Standard R 1 49. 1 - 1 972) 

ER-I - Household Electric Ranges (American National 
Standard C7 1 1-1972) 

ER-2 - Household Electric Ranges with Class/Ceramic 
Groking Tops (American National Standard C71. la- 
1975) 

ER-3 — Cleaning Performance of Household Electric 
Ranges with One or More Pyrolytic Self-Cleaning 
Ovens (American National Standard C7I , lb- 1975) 

HLD-1 — Standard Methods of Measuring Household 
Tumble Type Clothes Dryer Performance (American 
National Standard A197. 6-1975) 

'HLD-2EC — - Test Method for Measuring Energy Con- 
sumption of Household Tumble Type Clothes Dryers 

HI.W-1 •— Performance Evaluation Procedure for House- 
hold Washers (American National Standard 7.224.1- 
1971) 

•HLW-2EC — Test Method for Measuring Energy Con- 
sumption of Household Clothes Washers 

HRF-2-ECFT — Test Procedure to Determine the Freezer 
Temperature and Energy Consumption of Household 
Refrigerators, Combination Refrigerator-Freezers, and 
1 lousehnld Freezers 

DW-1 Household Electric Dishwashers (American 
National Standard A197. 5-1975) 

HU-1 — Appliance Humidifier Standard (American 
National Standard Z235. 1-1972) 


IIU-IA — Humidifier Application Standard 

RAC-1 — Room Air Conditioner Standard (American 
National Standard 7.234. 1-1972) 

RAC-2SR — Room Air Condi! inner Sound Rating 
Standard 

DW-2PR - Plumbing Requirements for Household 
Dishwashers (American National Standard A 197. 1- 
1973) 

HEW-2PR — Plumbing Requirements for Home imtndrv 
Equipment (American National Standard A 197.2- 
1973) 

FWD-2PR — Plumbing Requirements for Household 
Food Waste Dis|x>ser Units (American National Stand- 
ard A 197.3- 1 973) 

FWD-! Standard Methods ol Measuring Household 
Food Waste Dis|x>ser Performance (American National 
Standard A 197.4-1974) 

TC-I - Pcrlormancc Evaluation Procedure for House- 
hold Trash Compactors (American National Standard 
A I97.7-I97(i) 

*CO-l — Household Electric Can 0|xmicis 

*CM-I Household Electric Coffeomakers 

*FB-1 — Household Electric Food Blenders 

*FP-1 - Household Electric Frv Pans 

*1-1 — Household Electric Irons 

T-l — Household Electric Toasters (ANS C70.6-1977) 

•WBSG-I - Household Electric Waffle Bakers and 
Sandwich Gritls 


STANDARDS UNDER DEVELOPMENT 


Performance Evaluation Procedure for Microwave 
Cooking Appliances 

Room Air Conditioner Sound Application Standard 
Household Electric Hair Dryers 

Household Electric Chafing Dishes, Fondues, Deep Fat 
Fryers, Saucepans and Dutch Ovens 

Household Electric Food Mixers 
Dishwasher Drying Performance Standard 

Test Method for Measuring Energy Consumption of 
Electric and Gas Ovens and Surface Units and 
Microwave Ovens 

Room Air Gmditioner Hours of Operation 
'Pending approval as American National Standard 


Household Electric Slicing Knives 

Refrigerator-Freezer Standard and Recommended Levels 
of Performance 

Sound Measurement and Rating Standard for All 
Appliances 

Method for Measuring Energy Consumption of House- 
hold Dishwashers 

Household Electric Curling/Styling Wands 
Household Electric Slow Cookers 
Household Electric Toasters/Broiler Ovens 
Household Electric Clocks 
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Second, the Association also develops safety standards but these are called 
"safety recommendations" because they are developed for the sole purpose to recommend 
their consideration for use by the independent safety testing laboratories, Under- 
writers Laboratories Inc., and the American Gas Association Laboratories. 

Everything done by a trade association because of participation by competitors is 
sensitive to antitrust laws. Thus participation in these activities must be voluntary. 
Those who participate in standards development work do so voluntarily and anyone has 
the choice of using the standards in whole or in part or not at all. 

Virtually every electric home appliance manufactured in the United States is 
tested and listed by Underwriters Laboratories Inc., and gas appliances are tested and 
approved by American Gas Association Laboratories to assure that these products are 
prudently safe. AHAM recommends that its members submit their products to these in- 
dependent laboratories rather than "marking their own papers"; manufacturers should 
have experts besides their own in appraising the safety of their products. 

I will not describe the detailed process of developing standards within AHAM and 
the steps standards must pass through all the way to its Board of Directors before 
they can be AHAM standards. The Policy and Procedures Governing Standards state that 
once this is accomplished, standards must be approved by all interested parties out- 
side of AHAM through a consensus process and ultimately be recognized as American 
National Standards. The safety recommendations to the independent safety laboratories 
proceed through their consensus procedures until they too are recognized in some form 
as American National Standards. 

THE CHANGING USES OF STANDARDS 

Having provided this brief background, it will be interesting to note how the 
role of product standards has changed until their present day use. Fifteen years ago 
standards, both for performance and safety, were used solely for the purpose of pro- 
viding manufacturers with a common laboratory test for the evaluation of their prod- 
ucts. Using these procedures a manufacturer determined how his product and any stage 
of its development ranked with his competitors and knew that his competitors deter- 
mined the performance levels of their own products along with their mutual competitors 
the same way to arrive at the same results. This use of home appliance product stan- 
dards has not changed and will not change in the foreseeable future. 

With additional uses these standards have become more sophisticated, more gener- 
ally recognized and more universally used by government, both at the federal and local 
levels, by consumer organizations, by consumer magazines, and generally by anyone who 
uses or prescribes the testing of a product. Although engineering lab tests 15 or 
more years ago were used in developing a product which was acceptable to the consumer, 
engineering tests had to become more consumer oriented. These test procedures had to 
be developed so that the demands on the product in the laboratory were more directly 
comparable with the demands to which the product was exposed during consumer use. 

Until recently, standards provided an adequate means for determining energy con- 
sumption of home appliances when tested in a laboratory. Often a product ranking was 
all that was necessary. With energy now a very serious issue, these laboratory test 
procedures must more realistically portray ways a consumer uses the product or be 
directly correlative to the results of the energy use of that product while being used 
by the consumer. This will be dealt with in more detail later in this paper. 

AHAM'S CERTIFICATION PROGRAMS 

These standards, particularly the standard methods for measuring performance, are 
directly applied to AHAM's certification programs. 

Through the use of an independent laboratory under contract to AHAM, AHAM veri- 
fies manufacturer certified ratings of the cooling capacity of room air conditioners. 
Until very recently, the energy efficiency ratio of room air conditioners was listed 
along with the certified cooling capacity and the certified watts of each room air 
conditioner but the regulations developed by the Department of Energy (DOE) have 
brought about at least temporarily a conflict which can no longer permit publishing 
these energy ratings. The reason for this will be made clear when we describe the 
DOE sampling plan. 

The cubic foot refrigerator volume and freezer volume and the shelf area of re- 
frigerators and freezers and until recently the energy consumption were certified 
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under the AHAM Refrigerator-Freezer Certification Program. The output capacity in 
gallons of water per 24 hours for humidifiers is certified and the water removal capa 
city rating in pints per 24 hours for dehumidifiers is similarly certified. 


To enter the certification programs, a manufacturer or private brand labeler signs 
a contract with AHAM which requires among many other things that he purchases certifi- 
cation seals, see Figure 2, and affixes one to each of his products. 


FIGURE 2 
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If he enters into any one of these four certification programs, every product that 
he manufactures covered by that program must be certified. He cannot certify certain 
models of room air conditioners: for example, without certifying others. It may be 
interesting to note that the purchase price of these seals covers the total cost for 
the operation of this program, the administration of the program, the testing, etc. 

GOVERNMENT USE OF HOME APPLIANCE STANDARDS 

The General Services Administration (GSA) references AHAM standards in its pro- 
curement specifications for all government purchases of home appliances. State gov- 
ernment agencies also require that certain products purchased be tested according to 
their respective AHAM standards. 

Any organization evaluating a product or a component, especially today, that has 
a desire to reduce the energy consumption to make home appliances more efficient can 
only determine the authentic efficiency improvement or reduction in energy use by ap- 
plying that product to these nationally and often internationally recognized standards. 

AHAM standards for the measurement of energy were developed as a result of very 
complex field testing of home appliances under actual use in hundreds of homes. 

The Department of Commerce's (DOC) voluntary energy program developed some four 
years ago provided that the energy consumption measured according to these standards 
appear on a label with the product at the point of sale. The label contained both the 
AHAM certification seal and the DOC program seal on room air conditioners. In addi- 
tion to room air conditioners, the program had advanced to the extent of formalization 
for refrigerators and freezers also using the AHAM standards when in December, 1975, in 
spite of the fact that the voluntary program had gotten under way in short order and 
was already working very well, the Energy Policy and Conservation Act was passed. 

This Act required that by 1980 the energy efficiency of appliances had to improve 
at least some 20% over the energy efficiency values of 1972 and that these home appli- 
ances had to be labeled with their annual cost of energy and that standards had to be 
developed to measure these energy efficiencies and their annual energy consumption. 

Over two years later, mostly as a result of bureaucratic bungling some of which I 
will describe because of its interest to people with knowledge of statistics and qual- 
ity control methods, has provided no tangible results— products still are not labeled 
with the energy values although the DOC voluntary program was effectively working for 
the purpose of energy conservation and provided the opportunity for consumers to save 
energy and money. The passing of the Act in December, 1975 formally suspended the 
voluntary program. 

Last year our administration in Washington proposed amendments to this Act. 

Whether good or bad, these amendments (not legislated at this writing) have further 
delayed the implementation of the program. My guess is it will be at least another 
two years until the program becomes fully effective. 







THE DEPARTMENT OF ENERGY SAMPLING PLAN 

The test procedures finally adopted by the DOE are indeed the AHAM test proce- 
dures with little if any change. For a number of products, room air conditioners and 
refrigerators particularly, the tests are very cumbersome. A single test for refrig- 
erators takes weeks although a number of refrigerators can be tested simultaneously. 

This sounds like a condemnation of the AHAM standards. It is not. These AHAM 
test procedures are very precise and closely controlled tests that are used in the 
development of a product and for random quality control production testing on an in- 
frequent basis to determine precisely the energy consumption of these products. Manu- 
facturers have developed and use simpler test procedures, shortcuts if you will, to 
determine quickly if the performance of their product would correlate with the perfor- 
mance if the same product were tested according to the complex AHAM test procedure. 

In some cases these shorter tests can be conducted on every product as it nears the 
end of its production line. 

When these test procedures were submitted both for the use in the DOC voluntary 
energy program and to meet the requirements of the Energy Policy and Conservation Act 
of 1975, industry expected as was the case with the DOC voluntary program that these 
test procedures would be used for exactly what they are intended for--the precise 
method of measurement. 

The DOE, then the Federal Energy Administration ( FEA ) , when publishing the test 
procedures added to the AHAM test procedure a sampling plan. 

The DOE sampling plan states that 

" (f) Room air conditioners. --Cl ) Except as provided in paragraph (f)(4) 

of this paragraph, no manufacturer, distributor, retailer, or private 
labeler of room air conditioners may make any representation with respect 
to or based upon a measure or measures of energy consumption described in 
§430. 22(f) unless a sample of sufficient size of each basic model for 
which such representation is made has been tested in accordance with ap- 
plicable provisions of this subpart such that, for each such measure of 
energy consumption, there is 'a probability of not less than 0.95 that 
the means of the sample is within +5 percent of the estimate of the 
true mean of such measures of the basic model." 

This sampling plan was the one dictated by the DOE for every appliance regulated 
by the Act with only slight variations in the +5 percent of the estimated true mean. 
Where the DOE has decided that the coefficient of variation is wider and more diffi- 
cult to control as it is for refrigerators, for example, +10 percent is allowed; there 
are very few such exceptions. 

At least three samples of every basic model must be tested and they must be 
tested using the complex AHAM test procedure, no correlative or shorter test procedure 
may be used. 

The industry's major objection to this sampling plan is the fact that it does not 
control product energy consumption variation; it requires only that sufficient units 
be tested until the mean of the sample is within +5 percent of the estimate of the 
true mean of total production. 

Where energy consumption is certified under the AHAM program for room air condi- 
tioners and refrigerator-freezers, a tolerancing system is prescribed which will not 
permit any unit that is produced to consume more than 110 percent of its certified 
energy value. 

Figure 3 is a comparison of technical issues between the appliance industry's 
tolerancing system for ratings and the FEA's sampling system for determining ratings. 
The conflict between the DOE sampling plan and the tolerancing system used by the 
AHAM certification programs is the basis for the need to stop certifying and listing 
in AHAM Certification Directories the energy consumption for room air conditioners. 

As Figure 3 indicates, among other things, the value arrived at by the AHAM toleranc- 
ing system could very well be different from the value arrived at by the DOE sampling 
plan. The DOE regulation states in effect in §430. 24(f) that no energy representation 
may be made for a product unless it is based on the test procedure and sampling plan 
issued by the DOE. The energy consumption value arrived at by the AHAM certification 
program and the one arrived at by the DOE sampling plan may be sufficiently different 



If it meets the DOE requirements , it would not meet the requirements of the AHAM cer- 
tification program and more importantly to manufacturers, because of the federal law, 
the situation could very well be vice versa. To make things worse, the requirements 
of the California state law are indeed at the moment different from those required bv 
the federal law. H 

FIGURE 3 
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be the average). 
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when establishing ratings, correlative 
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can be used. Correlative testing Is an 
Inherent part of the tolerarclng system 
because this system does not restrict the 
method of establishing the rated values. 
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dures to be certain that the rating Is 
maintained because the rating has only 
an upper end limit. 


6. The tolerance system allows manufacturers 
to select rating based on testing and 
experience and allows him to adjust the 
test value based on any anticipated vari- 
ability In production. 


7 . Toleranclng system allows manufacturers 
to conservatively rate product because 
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3. Assures only that the average of all 3. Relates to #2 l.e. , possible 
units will be within a specified limit consumer deception, 
of the population mean. (Assuming the 
sample tested Is truly representative 
of production.) 


4. Correlative testing Is presently not 
allowed and would have to be specifi- 
cally Included for establishing 
ratings. 


For room air conditioner and 
refrigerator-freezer, lack 
of correlative testing Is an 
economic hardship because of 
expensive facility require- 
ments (especially for room 
air conditioners) and long 
duration of test (especially 
for refrigerator-freezers). 
This does not apply for most 
other AHAM products. 


5. Sampling plan does not lend Itself to 5. 
Q.C. procedures because it addresses 
itself to the total population mean. 


6. Sampling plan Is an Inaccurate method 6. 
of arriving at the rating because It 
assumes that the sample (which must be 
early production or pre-production) Is 
representative of all products that will 
be produced and contains all the 
variances likely to be encountered during 
production of that model. (All these 
assumptions are known to be false for 
room air conditioners.) 


Manufacturers must have a 
Q.C. system to market an 
acceptable product. Applying 
the sampling plan technique 
to Q.C. would be an added 
burden (particularly to room 
air conditioners and refrig- 
erator-freezers and to vary- 
ing degrees to other products). 

Sampling plan may result In 
■Inaccurate ratings (partic- 
ularly true for room air 
conditioners and refrigerator- 
freezers and to a lesser 
degree to other AHAM products). 
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mean of sample. It does not provide 
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Room Air Conditioner Industry’s 
Specific Concerns 

Room air conditioner manufacturers 
do not have confidence In the 
rating established under the 
sampling plan because: 

a) It requires rating at 
the mean (a very rigid 
number) , 

b) It does not provide a 
system for adjusting the 
Initial rating as produc- 
tion variation changes, 

c) additional testing after 
establishment of initial 
value (although not 
required) could cause 
Initial rating to become 
suspect as wrong, 

d) In order to determine If 
the initial rating Is 
right or wrong a signif- 
icant number of units 
(30 to 40) would have to 
be tested. 


Aside from the fact that the DOE sampling plan has no requirements on product 
variability with regard to energy consumption, a product must be rated before produc- 
tion actually begins or certainly before any energy representation is made and the 
sampling plan assumes that the sample tested is truly representative of production. 

On products such as home appliances I can assure you that a product manufactured dur- 
ing a year or two year period can have some variation in the measured energy use. The 
AHAM certification program provides a realistic tolerance; over its full range the 
consumer is protected fairly. The DOE program does little, if anything, to control 
this variation and once a product's energy consumption is so rated there is no pro- 
vision to change this rating regardless of any variation that may occur throughout 
the period the model may continue in production.. After your analysis of Figure 3, 
we'd be interested in your evaluation of the two systems. 

All of this may be similarly true for refrigerators but at this writing, because 
the complex testing is of such long duration some manufacturers do not have test re- 
sults for all of their models. Thus it i.s not known whether there is a conflict with 
numbers nor are the numbers available for publication in the Certification Directory if 
there will be no conflict. 

Until the differences between the two programs can be resolved, any valuable pur- 
pose that the AHAM certification programs provided the consumer has passed on to obli- 
vion. The DOE has been informed of all of this and is not willing to bend or change 
its position. With energy a critical issue the government has stopped two important 
programs which were in place serving the consumer with vital energy information for 
four years. 


THE VOLUNTARY STANDARDS SYSTEM IN THE UNITED STATES 

As you are aware, the procedure used by the Association of Home Appliance Manu- 
facturers for developing standards requires a consensus agreement by all interested 
parties outside of AHAM so that these standards can be recognized nationally and inter- 
nationally as American National Standards. This procedure varies, little from those 
used by other voluntary standards developers. Until now, whenever any government 
agency had a use for these standards, instead of developing new and different ones, 
it recognized standards that were developed by the experts, and it has been generally 
recognized that the expertise does indeed lie within the private sector of industry. 
Certainly no one understands home appliances more intimately than the engineers who 
have designed, developed, tested and manufactured them. 

I believe that those of you who are involved in the voluntary standards develop- 
ment system are aware of the fact that the government wants to take over, set up rules 
and regulations, making it practically impossible for an organization such as AHAM to 
continue to develop good standards for home appliances, universally recognized stan- 
dards. 

You should be familiar with the content of Senate Bill 825, The Voluntary Stan- 
dards and Accreditation Act of 1977, "a bill to foster competition and consumer pro- 
tection policies in the development of product standards, the testing and certifica- 
tion of products, and for other purposes" introduced March 1, 1977 by Senator Abourezk. 

Very briefly this proposed legislation: 

1. provides for the Federal Trade Commission (FTC) to develop guidelines 
which all standards development organizations must follow; 

2. establishes an Institute of Standards within the National Bureau of 
Standards (NBS) to act as an advisor to FTC and to write standards 
if not done elsewhere according to requirements determined by FTC; 

3. places the DOC in charge of international standards participation 
by the United States; 

4. establishes a National Laboratory Accreditation System; 

5. establishes a National Standards Management Board to "oversee" all 
standards activities in the United States. This replaces the 
American National Standards Institute (ANSI). 

S.825 was not enacted last year; whether it might be enacted in the future is now 
only of academic interest because various government agencies are adopting the basic 


policies and procedures spelled out in S.825 such that any standard developed by the 
private sector must prescribe to the rules spelled out in S.825 if they are to be used 
by a government agency. 

In July, 1977, the DOC developed a lengthy report which it submitted to the 
Office of Management and Budget (0MB) which seriously impugns the existing voluntary 
standards and testing laboratories systems in the United States. The report has a 
section called "Horror Stories" related to standardization and certification activi- 
ties. Most of these "stories" were based on complaints from parties with self-serving 
interests, most were not investigated and were exaggerations, not factual or contained 
only innuendos. 

The DOE now has a public law on its acceptance of standards developed by the pri- 
vate sector. This public law includes again most of the requirements of S.825. I 
have been called by DOE lawyers on occasion who are considering the use of AHAM stan- 
dards and have been asked questions such as to identify, for example, the distributors 
or retailers who participated in the standard development committees. It is very dif- 
ficult to explain to these lawyers that a distributor or dealer is not interested in 
spending their time developing product standards because their time can only be used 
productively selling their product. This is a fact of life and something that I don't 
believe can be forced even by regulation. The only obvious alternative is the devel- 
opment of these standards by government and government alone and apparently that is 
the reason for imposing these requirements in the first place. 

Finally, the 0MB published in the January 3, 1978 Federal Register a proposed 
0MB circular on federal agencies interaction with and use of voluntary consensus 
standards-developing bodies. This again imposes many of the requirements as proposed 
in S.825, The Voluntary Standards and Accreditation Act of 1977, and I believe that 
unless this circular is changed substantially, there is no way that AHAM, for example, 
as a voluntary standards developer can, no matter how intent it may be to meet the 
requirements of this circular, continue to be a developer of home appliance "volun- 
tary" standards. 

The very same standards that government found acceptable are no longer acceptable 
because they may not have been developed according to a new set of government regu- 
lations. 
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THE JCPENNEY COLOR-CODING PROGRAM FOR TEXTILES 


Robert A. Lewis 
Merchandise Evaluation Engr. 
JCPenney Co. , Inc. 

1301 Ave. of the Americas 
New York, N.Y. 10019 


INTRODUCTION 

The purpose of the color-coding system is to minimize customer returns for 
reasons of color variations (off shade, too weak, too heavy) of curtain, drapery, 
and bedspread ensemble items available through the JCPenney catalogs. 

Our suppliers compare and evaluate all new production of dye/print lots against 
an approved standard and have been instructed to ship to us only the merchandise that 
falls within the established limits of tolerance of the color-coding system. 

DESCRIPTION OF THE COLOR-CODING SYSTEM 

The color-coding system as devised and implemented by JCPenney is based on the 
visual, subjective comparison and evaluation of new dye/print lots of textile fabrics 
versus a standard under controlled lighting conditions to determine their degree of 
sameness or difference with respect to shade and strength. 

New lots of white, dyed or printed fabrics of the same style, color/design are 
evaluated against an approved standard and are rated acceptable or not acceptable 
within the established limits of tolerance. Unacceptable merchandise received at a 
JCPenney Catalog Distribution Center is returned to the supplier. 

The color-coding procedure is comparatively simple and straightforward to carry 
out. The range of acceptable tolerances of new dye/print lots to standard is quite 
broad and reasonable. 

The technique for the color-coding of textiles is similar to that used for the 
evaluation of colorfastness tests (light exposure, washing, dry cleaning, etc.) 
where the test sample is visually compared to a standard for degree of shade change 
and is assigned a numerical rating. 

Individuals involved in the color-coding of fabrics must have an aptitude for 
color judgement and be free from color deficiencies. There are testing methods 
available for determining a person's color aptitude and/or color deficiency. 

The in-process controls and standardization methods employed by most dye houses 
provide a flow of dyed and printed goods that fall well within the limits of 
tolerance of the color-coding system and can be readily coded when received by our 
suppliers. 


PROCEDURE FOR COLOR- CODING 


The Standard 50 

The approved standard shade - white, dyed or printed, is arbitrarily assigned 
the code number 50. 

The Standard 50. is placed on a flat surface with a neutral grey background under 
the recommended (artificial daylight) viewing light. 

The new dye/print lot to be evaluated is placed alongside of (abutting but, not 
overlapping) the Standard 50 . 

The Evaluation 

The comparison of the new lot with the Standard 50. and assignment of the color- 
code number. 

t> 1978 American Society for Quality Control 
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Color-Code 50 - - - - 

New lots that are found to be the same shade and strength as the Standard 50_ (or 
with just a nuance of difference) are coded 50 . 


slightly weaker in strength but, on shade to the 
items coded 50 and be acceptable to most customers) 


Color-Code 5_5 - - - - 

New lots that are found to be slightly heavier in strength but, on shade to the 
Standard 5_0 (could be shipped with items coded 50 and be acceptable to most customers) 
are coded 55 . 

Code UC (Unacceptable) - - - - 

New lots that are found to be noticeably weaker in strength to the Standard 50_ 
(fall outside the limits of tolerance, rate below 45, could not reasonably be shipped 
to a customer with items coded 50) cannot be color-coded and are therefore unaccept- 
able . 


Color-Code 45_ - - - - 

New lots that are found to be 
Standard 50_ (could be shipped with 
are coded 45. 


New lots that are found to be noticeably heavier in strength to the Standard 50_ 
(fall outside the limits of tolerance, rate above 55, could not reasonably be shipped 
to a customer with items coded 50 ) cannot be color-coded and are therefore unaccept- 
able . 


New lots with noticeable variations in shades (redder, yellower, bluer, etc.) 
to Standard _50_ are considered to be mismatched shades that cannot be color-coded and 
are therefore unacceptable. 

Note: Trim - fringe, tassels, braid, rickrack, embroidery, etc. are also evaluat- 
ed for conformity to standard. The overall color of a fabric may be satisfactory 
and codeable but, not be acceptable due to noticeable and objectionable shade 
differences in the applied decorations. 

The color-coding method is applicable only to those fabrics of the same fiber 
composition, construction, color and finish. Related items of similar color and 
design but, of different fiber and construction are offered as coordinates, not as 
true shade matches, the only prerequisite is that they be reasonably close in shade 
and strength as to harmonize satisfactorily within an ensemble. 

Color measurement by instrumentation (colorimetry - spectrophotometry) can be 
used as an adjunct to the color-coding system but is limited almost exclusively, to 
the evaluation of solid shades and not prints or other multi-colored fabrics which 
account for over 80% of all textiles being coded. 

COLOR- CODING TECHNIQUES 

1. Fabrics under evaluation should be perfectly flat, free from wrinkles and 
folds. 

2. Selvedges should be removed if they interfere in any way with the evaluation. 

3. Swatch size should be at least 9X12 inches for solid shades. The size of 
printed fabrics should be large enough to contain all colors and elements of the 
design. 

4 Check to see that both the standard and the new lot are face up; this will be 
obvious for printed fabrics but, sometimes in solid shades, differences are not as 
readily detected between the face and back of the fabric. 

5. Align both the standard and the new lot in the same direction, i.e.: warp 
running lengthwise and the weft (filling) running horizontally. A ribbed fabric, 
such as a twill, should have the ribs running in the same direction. 

6. Printed Fabrics - Printed fabrics of multi-colors and with intricate designs 
by their nature tend to camouflage shade differences. In evaluating a printed fabric 
or any cloth containing a design, the predominant feature (the background or the 
design) is the first to be compared to the standard. The secondary feature is then 
evaluated. Any unusual element of the design itself - a brilliant scarlet flower 


for instance, that stands out in the overall print should be evaluated very carefully. 

7. Sheer fabrics should be folded over so that the transparency of the material 
will not lead to erroneous results; a thickness of as many as eight layers may be 
necessary. 

8. Plush fabrics are especially tricky to evaluate. The same fabric can appear 
to be completely different in shade (lighter-darker) depending upon the position in 
which it is placed and the way the light is reflected from the surface. The fabrics 
may have to be moved into various positions until a satisfactory alignment is- 
obtained. 


9. Metamerism, (apparent color changes caused by different sources of light) can 
be determined by changing the light source momentarily to incandescent/fluorescent but 
the color-coding itself is carried out under the daylight light only. 

10. The presence of an optical brightener can be determined by examining the 
fabric under ultraviolet light. White fabrics are more likely to contain an optical 
brightener. If a new lot of white goods appears to be "whiter" than the standard or 
vice versa, it may be due to more complete bleaching of one fabric than the other 

or one may contain more optical brightener that the other. The ultraviolet light will 
readily establish this fact. 

ORDER FILLING BY COLOR-CODING NUMBER 

The suppliers have been instructed to place the color-code number on all in- 
dividual packages of merchandise that they ship to our distribution centers. 

Whenever feasible, the customer’s orders are filled by picking from the bins, 
merchandise with like code numbers, i. e; 45_ with 45 , J50_ with 50 , and 55 with 55 . 

If this is not possible then combinations of items coded 45 and 50 or 50. and 55 
are picked for shipment to the customer. 

Exception - curtain and drapery panels (where two or more may be hung side by 
side at a window or door) must have the same color-code designations. 

Because of the very noticeable shade depth differences, items coded 4_5. and 55_ 
in no instance can be shipped together on the same customer order. 

LIGHTING REQUIREMENTS FOR COLOR-CODING 

Artificial daylight is the recommended light source for carrying out all color- 
coding evaluations. The use of the same illumination in all color-coding areas en- 
sures closer correlation and agreement of results among the various evaluators. 

The necessary lighting equipment is available from the Macbeth Corporation, the 
Diano Corporation as well as several other manufacturers of similar units. 

Booth mounted installations provide a standardized viewing environment; therefore 
no consideration need be given to the surroundings. However, when a suspended over- 
head illuminator is used, it is very important that certain precautions be taken to 
ensure optimum results. Extraneous illumination (natural and artificial) must be avoided. 
The best location for the lighting unit is a windowless room. If the lamp is used for 
the lighting of a localized area, then this area or table should be shielded from 
extraneous illumination by a curtain. In this instance the curtain must be of a light 
neutral grey shade which in addition to shielding the special area from extraneous 
illumination, will provide a second important factor, a neutral grey (Munsell H-8 
grey) background which is desirable to avoid brightness contrast and color reflections 
which might result if the curtain were anything but neutral. (For example, a violet 
sample reviewed in a blue surrounding will appear redder; a violet in a red surrounding 
bluer than it would appear in a neutral grey surrounding). All extraneous articles 
must be kept out of the color-coding viewing area at all times. 

FABRIC STANDARDS - MAINTENANCE AND HANDLING 

As recommended to JCPenney and supplier personnel, a file of approved reference 
standards of all dyed/printed curtains, drapery and bedspread ensemble items offered 
in our catalogs shall be maintained. 
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Since most fabrics age and change color over a period of time, reasonable care 
must be taken in protecting them from atmospheric degradation due to moisture, light, 
heat, fumes, dust, handling, etc. Standards should be stored in a thoroughly pro- 
tected environment . 

Suggested Storage Procedure: 

One large fabric piece, minimum of 1 yard, coded 50 and identified as MASTER 
STANDARD, should be folded and placed in a plastic bag, (black preferably ) . The 

flap of the bag should be sealed with a strip of adhesive type tape. A label con- 
taining pertinent information should be affixed to the bag. 

A smaller swatch of the standard (the working standard, used for routine in- 
spection) should be kept in a separate plastic bag so that each time it's needed for 
color-coding, the master standard is not disturbed. 

To reduce the possibility of soiling the standard swatches (oil from hands etc. ) 
the swatches should be handled by means of a folded cardboard (file card) stapled to 
one side, or by clean disposable gloves. 

For quick retrieval, the fabrics should be filed alphabetically by supplier, and 
in turn by style, color and catalog number. 

The storage cabinet should be located in an area removed from any direct heat 
source . 

Checking Standard for Color Change 

Periodic .comparisons should be made of the working standard and the master 
standard. If the working standard is found to have faded, become soiled or has changed 
color in any way, as compared to the master standard, it should be destroyed. A new 
cutting properly identified is then taken from the master standard for use in the 
routine evaluation. 

To ensure that the master standard has not changed, colorimeter measurements 
should be made at the time of initial selection. The colorimeter measurements can be 
filed. At a later time the standard can be retested. If it is determined that the 
standard has faded or changed color in any manner, it can be replaced with a new 
one that corresponds with the original colorimeter measurements. 

CONCLUSIONS 

The color-coding system has eliminated obvious shade differences in merchandise 
that most certainly would have caused dissatisfaction if received by the customer. 

The system has been in effect for over two years and it is accomplishing what it 
was designed to do - cut down the number of returns for reasons of shade variations, 
and in consequence has reduced unnecessary expenses and increased customer satis- 
fact ion. 

As with any plan involving a number of people, cooperation is needed by all 
participants to make it maximally effective. Adherence to the requirements of the 
color-coding system will be mutually advantageous to the suppliers, to the JCPenney 
Company, and to the customer. 
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QUALITY CONTROL IN THE UPHOLSTERY INDUSTRY 


T. Hugh Talley, Director 
Research and Development 
Schnadig Corporation 
Montoursville , Pennsylvania 17754 


There are several descriptive statements which relate to the upholstered 
furniture industry: 

1 - Historically, our industry has been a fashion-style-design oriented 
industry, not a technically oriented industry. 

2 - In general, definitive, quantitative programs for the monitoring and 
assurance of quality-of-product have been virtually non-existent. 

3 - To a great extent, the upholstery industry depends on its vendors for 
research and development. 

4 - To a great extent , the upholstery industry accepts the "word of their 
vendors" concerning the quality of the materials the vendor delivers. 

5 - Uniqueness of product doesn't last very long because of the "knock-off" 
concept; defined as "another company copying your design and then cheapening it" 

6 - The industry is rife with small, low-overhead companies, who lack any 
quality consciousness, but who provide the same "aesthetic-look" at much lower 
prices . 

7 - With respect to "product knowledge", upholstered furniture is one 
consumer product about which our consumers - and retailers - know very little. 

Within the last five to ten years, a minor revolution has occurred in our 
industry. The revolutionary changes can be attributed to many factors, but here 
are a few of the more important ones: 

1 - The germination and subsequent growth of "The Nader Movement". 

2 - The increase in the amount of information for the consumer; consequently, 
our consumers are becoming better educated, more sophisticated buyers, and more 
quality and performance oriented. 

3 - The growth of involvement of federal and state governments in 
consumer-product quality and performance. 

4 - The hard-fact realization that programs which accurately monitor and 
assure product-quality also reduce cost and can increase profits! 

Our company, Schnadig Corporation, recognized the needs many years ago, 
and then began to plan for implementation of a viable, thorough, and quantitative 
program for product-quality monitoring and assurance. At Schnadig this program 
is now called "Total Quality Assurance" (TQA). Our TQA program was originally 
complicated, somewhat, because our company has six upholstery plants scattered 
throughout the United States - all making essentially the same products. We 
also have three woodworking facilities in three different geographic locations. 

TQA began with the realization of all the facts previously mentioned, and 
then TQA was implemented in three major areas: 

A - PSYCHOLOGICAL 
B - TECHNICAL 
C - ADMINISTRATIVE 
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PSYCHOLOGICAL 


In our opinion, a quality program based solely on numerical and administrative 
entities will not work adequately. Quality psychology must permeate the entire 
organization; thus, TQA must begin and originate in the "board-rooms" and then flow 
unabated throughout the entire organization; then it will flow to our customers as 
another "end-product" of our company. The chairman of our board and corporate 
executive officers initiated and actively supported the "TQA attitude" and, ladies 
and gentlemen, regardless of how idealistic it may sound, the bottom line is very 
accurately simple it works! 

Just a side comment on idealism - it is high-time that our society returned to 
some "old-fashion", maybe "corny" idealism — instead of the phony pseudosophisti- 
cation that permeates our society today! Quality is an ideal; the psychology of 
quality should be idealism; and deviations should arise or occur only from the 
ideal - not from a compromise! 


TECHNICAL 

The technical part of our TQA program can best be described by two statements, 
in order: 

1 - If the inside of a piece of upholstered furniture is made correctly, 
according to original, engineered specifications, the outside - including aesthetics 
and comfort - will inexorably be correct. 

2 - Put a number on it! (Or at least try to.) 

The technical part of our TQA program began by separating the needs in two 
areas, i.e. the control of the products we purchase (our raw materials) and the 
control of the processes we use to assemble the raw materials into our final 
product . 

We found that our major problem was in the quality of our raw materials, 
particularly , in foams and padding materials, springs, and upholstery fabrics. 

Our next step was to develop first-generation specifications in the major-problem, 
raw-material areas . We then developed test methods in areas where none existed 
and used conventional test methods where supplier-user agreement could be attained. 

We then developed raw-material sampling programs geared to yield data which 
when properly, statistically analyzed would: assist us in vendor performance 
rating; guide us in developing better specifications with reasonable tolerances; 
and guide us in the derivation of future sampling programs for routine, daily 
sampling. 

The next step was the most complex step, i.e. the relation of all the infor- 
mation which we had previously developed to aesthetics and comfort. 

With respect to "comfort" , our sales and marketing group is attuned to our 
particular customer's definition of comfort and comfort ranges. Ranges of "comfort" 
were determined subjectively by our sales and marketing group, e.g. middle range of 
"just-right"; low end of good comfort range; high end of good comfort range; and so 
on. These samples were then taken to our labs where the measurable characteristics 
of the whole pieces and, after disassembly, the measurable characteristics of the 
components were determined, e.g. load- vs . -deflection , fabric breathe ability, fabric 
stretch, spring loading, spring deflection characteristics, etc. 

Once the measurable characteristics of the components were known - with their 
ranges - and once these characteristics and ranges were related to the whole, 
assembled piece of furniture, the next task was to relate this information to the 
specifications and tolerances determined in the first survey on raw materials. 

We then contacted our vendors and outlined our needs as far as specifications 
and tolerances were concerned, and when our vendors saw documented evidence of our 
needs, cooperation was a matter of natural course. 

Many of our raw materials, such as foams and springs, are used in vast volumes 
with exceptionally fast inventory turnover. These products had to be screened and 
tested in each individual plant. We designed and built the equipment for testing 
these products in each plant, and once the equipment was installed and operating, 



statistical sampling plans were derived for each material at the volumes used in 
that particular plant. 

Other raw materials are used in such volumes and rate in importance such 
that they are routinely sampled, and the samples are sent to the central testing 
labs for analysis. 


Our TQA program on major raw materials includes: 


lumber 6 wood composites 
finished KD frames 
adhesives 
assembled frames 
plastic frame materials 


springs 
foams 
fabrics 
buttons 
deck fabrics 


Our TQA program on minor raw materials includes all other materials used to 
manufacture our product. 


The relation between raw material properties and final aesthetics is a little 
more difficult; however, remember the statement, if the inside is built properly, 
according to original, engineered specifications, the outside and final aesthetics 
will be correct. Attaining the "state of correctness" on the inside and thus the 
proper aesthetics outside is accomplished basically by two programs. The first of 
these programs involves training and educating our craftsmen in each step of the 
process by use of specification review meetings, photograph-drawing review meetings, 
and video tape sessions on the more complex steps or processes. For interest sake, 
permit me to interject here that Schnadig Corporation pioneered the use of in-house 
video systems (in our industry) for training our personnel. The second of these 
programs involves sampling and inspections of each step in the manufacturing process 
- from start to finish. Major areas of concern are outlined for the inspectors - 
as well as procedures for inspecting each step. 


ADMINISTRATIVE 


The routine administration of our TQA program involves: 

1 - Ascertaining the proper flow of information and data 

2 - Liaison with sales and marketing 

3 - Ascertaining proper follow-up on field complaints and problems 

4 - Planning the biannual meetings for individual plant TQA 
supervisors and production executives 

5 - Liaison between design and merchandising 

6 - Liaison between R £ D, plant engineering, and product engineering 

7 - Planning quality sessions with our customers 

8 - Liaison between top-level, executive management, and other departments 

I have, briefly and generally, outlined for you how our company has taken 
some giant strides forward in the area of quality assurance. We are cognizant 
of the fact that we are not yet as sophisticated or as statistically quantitative 
as many of your companies; but we are in the beginning stages, and we have shown 
a substantial "return" on our TQA investments to date. The "return" on our TQA 
investments more than justifies the present program, and these "returns" also 
justify program expansions and extension. Our goal is to, ultimately, have each 
raw material and each manufacturing process controlled by a sampling- inspection 
plan, which, by magnitude of confidence-interval, represents the importance of 
that raw-material or process to the total. 
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QUALITY COST AND PROFIT PERFORMANCE 


Francis X. Brown, Staff Assistant 
Roger W. Kane, Staff Assistant 
Westinghouse Electric Corporation 
Corporate Product Integrity 
Pittsburgh, Pennsylvania 15235 

ABSTRACT 

Use of failure cost (rather than total quality cost) as the principal financial 
measure of quality performance opens the door to more extensive and effective applica- 
tions of this management tool. The measure is better correlated with other indicators 
of quality performance; the demonstrated correlation with profit performance forms a 
rational basis for quality cost performance norms; and the projected effect of a specific 
quality improvement on overall business performance can be forecast with better confi- 
dence . 


BACKGROUND IN PIMS 

From its earliest beginnings, the quality profession has engaged in the pursuit of 
profitable excellence. The list of searchers is as long as it is illustrious - Shewhart, 
Feigenbaum, Juran, and Schoeffler. . .Schoeffler? - While not a member of the quality 
community, he and his colleagues at Harvard broke the barrier that had prevented a 
direct coupling of quality costs to measures of business performance. (!) Prior to the 
landmark Profit Impact of Market Strategies (PIMS) study, quality cost methods, in 
general, utilized targets and emphasized prevention expenditures to reduce failure 
costs thereby bringing total quality costs into balance. In 1951, Feigenbaum wrote: 

"An unprofitable cycle is at work that operates something like this: the more defects 
produced, the higher the failure costs. The traditional answer to higher failure costs 
has been more inspection. This, of course, means a higher appraisal cost . .. and 
the higher they go, the higher they are likely to go without preventive activity". (2) 

In the same book Feigenbaum says "... the product can be provided with those quali- 
ties which motivate purchase by the consumer and thus increase salability." As 
recently as 1970 Juran and Gryna recognized the problem of being unable bo measure the 
effect of quality on business performance with these words: "The facts on cost of 
quality are often precisely ascertainable. However, facts on value of quality are more 
nebulous. In particular, while the factors of quality reputation and customer goodwill 
are conceded to be of great importance, the present methods for evaluating them are 
quite primitive . " ( 3 ) 

And there matters stood until the PIMS work was published in 1974. PIMS con- 
cluded, at least qualitatively, that the influence of Quality Reputation on Return on 
Investment was very potent indeed and second only to Market Share. Here, at last, 
was an analytic tool to couple quality to profitability. 

The work reported in this paper was begun in 1975, and resulted from efforts to 
utilize the PIMS methodology to quantify the impact of quality on profitability. In 
order to achieve the quantification several criteria were established: 

(1) Provide a financial measure of quality performance that is consistent 
with other measures of quality performance; 

(2) Provide a measure which responds in a timely manner to changes in 
actual quality performance; 

(3) Provide a measure that is credible to Management, acceptable to the 
Controller organization, and suitable for inclusion in the routine 
financial reports of the business. 
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The primary data sources used for this study are the quality cost data reporced by 
Westinghouse divisions in response to a Corporate requirement, and financial state- 
ments of these same divisions, (coded in this paper). 

SEARCH FOR A MEASURE 

Initially our attention was put on total quality costs (TQC) , with the expectation 
that the best performing divisons would have the lowest TQC. However, a ranking of 
divisions by TQC did not agree with rankings based on other measures - such as the 
division’s reputation for quality, or the findings of Corporate Quality Program Audits, 

A better measure was found when attention was limited to failure (F) costs alone. 
Divisions with low failure costs were also found to have better reputation for quality 
performance . 

Figure 1 illustrates the performance of 3 similar divisions between 1975 and 1977. 
All three divisions introduced quality program improvements during 1975 which contri 
buted to lower failure costs in 1976 and 1977 - lower in dollars and lower as a per- 
centage of Value-Added Sales . 

Failure cost relative to value-added sales (F% VAS) was found to be a better 
indicator of quality performance trends than the more common measure of F% total sales 
billed, since V.A.S. is a better measure of opportunity for failure costs to occur. 
(Value-Added Sales is the difference between gross sales billed and the cost of pur- 
chased material or services.) 

Figure 1 shows not only that these divisions achieved a reduction in F% VAS, 
but also (as predicted by the PIMS study) that the protit performance of these divisions 
improved when failure costs were reduced. 



FIGURE 1 FIGURE 2 

Figure 2 shows the relationship when data for other divisions is added to the nine 
data points of Figure 1. The letter "A" on the vertical scale represents what was de- 
fined to be an acceptable level of profit performance. The data show good linear corre- 
lation (r= -0.90) between profit margin (M% sales) and failure costs (F% VAS). 

From this data the least-squares regression line was found to be: 

M = (A+13) - 3.6 F 

vas 

The observation that failure costs and profit performance are highly correlated does 
not in itself prove that either is a direct cause of the other. External factors such as 
an industry-wide change in demand for the product can cause both profit margin and 



F% VAS in a particular division to shift — even though no change may have 
occurred in quality performance itself. 

Whether or not a cause-and-effect relationship is present, the correlation of 
margin to failure costs can be highly useful - for example, to test the credibility of 
reported failure cost data. For the 64 data points shown in Figure 2, the difference 
between actual profit margin and the margin predicted by the regression mode has a 
standard deviation of 3.5 percentage points. The two parallel lines in this figure 
lie two standard deviations (seven points of profit margin) above and below the least- 
squares regression line, and therefore represent upper and lower boundaries of a 
95% confidence band for margin as a function of failure cost. 

When data points are found to lie outside the appropriate confidence band, they 
should be investigated. The few exceptions observed were investigaged, and in 
nearly every case this investigation revealed errors in the reporting of failure cost 
data - for example, use of estimates or accrual rates rather than actual failure costs. 

As mentioned earlier, some of the correlation between margin and failure costs 
may be the result of both factors responding simultaneously to a third variable. None- 
theless, there are also indications that a strong cause-and-effect relationship is 
present, and that a sustained program for reducing failure costs can make a major 
contribution to improved profit performance . 

THE MULTIPLIER EFFECT 

Figure 3 helps to demonstrate the profit impact of a one-dollar change in failure 
cost. (The same measurement base is used for both variables.) The least squares 
regression line for 64 data points becomes 

M = (A + 12.6) - 5.45 (F% Sales). 
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FIGURE 3 


Although the linear correlation for 
M:F% sales (r = -0.79) is not quite as 
good as for M:F% VA.S,/u;3e of the 
same measurement base (sales billed) 
reveals the existence of a strong 
"multiplier effect". The slope of 5.45 
means that profit margin (before taxes) 
improves an average of $5.45 for every 
$1.00 reduction in reported failure 
costs . 


Most managers are aware that some costs of quality failure are not easily 
identified as failure costs, but that they affect profit nonetheless: 


o in-process quality failures may cause parts shortages 
which create production downtime at subsequent 
operations; 

o some in-section rework is absorbed in productivity 
and reported as a labor variance; 
o chronic rework or excessive "shrinkage" may necessitate 
scheduling of overtime or purchase of additional production 
facilities; 





o completed product which fails, in final test 

may result in increased work-in -process inventory 
and reduced billings for the month; 
o product failures in the field can, at least in the 
long run, contribute to reduced market share or 
poor price realization. They may also contribute 
to past-due or uncollectible receivables. 

Although most managers recognize failure effects such as these (sometimes 
called "symptoms of failure disease"), it has not been common practice to quantify 
the expected benefit in these areas when attemping to justify capital expenditures 
for quality improvement. Some of the observed slope of 5.45 may be only coinciden- 
tal, but our experience indicates that a multiplier effect of at least three or four is 
directly related to such hidden effects of quality failure. 

FINDING OPTIMUM "BALANCE" 

This multiplier effect of failure costs on profits also explains why minimum TQC 
is hardly ever optimum . Reported quality costs are typically at their lowest when 
the voluntary costs of quality (prevention plus appraisal) are about equal to the 
reported failure costs — or 40% to 60% of TQC, as shown by the solid lines in 
Figure 4. If, however, the "true" cost of failure is found to be more nearly four 
times the reported costs, the point of true minimum cost would move further to the 
right, as shown by the broken lines of Figure 4. 



FIGURE 4 

Our data tend to confirm this. Reported TQC is at a minimum when prevention 
plus appraisal cost (P+A) constitute about half of the reported total quality cost 
(TQC), but profits continue to improve until (P+A) contributes 70% to 80% of the 
reported TQC. 


EVALUATING PROGRAM EFFECTIVENESS 

Since a particular level of profit margin (A) was defined as "acceptable", and 
since our data confirms the PIMS finding that divisions with better quality per- 
formance also earn more profits, we used this relationship to define an acceptable 
level of failure costs. The level of failure costs at which the M:F regression line 

crosses M = A % sales is described as F , and the level of failure costs at which 

par 



the regression line crosses M = (A~8) % sales is defined as F , . 

tol 

For the divisions included in Figure 2 data these points were found to be 

F =3.6% VAS and T l =5.8% VAS (See Figure 5) Divisions whose reported failure 
par tol * 

costs are less than or equal to F are considered "green" — shown as zone "G" in 
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Figure 5 . 
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FIGURE 5 

On the other hand, when reported failure costs exceed F^ the status is con- 
sidered "Red", and the division is suspected to have some form of "failure disease". 
We contact these "Red" divisions to determine which specific symptoms of failure 
disease might be present (low productivity, missed billings, loss of market share, etc) 
and begin to work with them in establishing a program for improvement. 

As you might suspect, when reported failure costs lie between F and F the 

p otir tol 

performance !.s considered "yellow". If left alone, yellow will usually change to red. 

FAMILY OF PROFILE TYPES 

Finally, we should say a word about the selection of divisions included in Figure 
2. Not all Westinghouse divisions fit this profile type, although it is the most common. 

Some divisions continue to report margin and failure costs which fall above and to 
the right of the data plotted in Figure 2. Typically these divisions also tend to be in 
higher risk divisions, with higher technology requirements for their products or 
services. Such divisions fit the profile types III or IV shown in Figure 6. 
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FIGURE 6 


Other divisions report margin and failure costs which fall below and to the left 
of the Type II profile divisions. In many cases, these divisions are engaged in a 
more mature business, with reduced technological requirements. However, the 
steeper slope of these Type I divisions also implies that fewer of the real costs re- 
lated to failure are identified and reported. (So long as the reporting is consistent 
from month-to-month, and the portion of failure costs which is identified and reported 
is a valid indicator of performance trends, this "under-reporting" need not cause 
concern.) 


Performance standards will of course differ for the various profile types. At 
present, we are using the same level of profit margin (M = A % Sales) to define F 

As shown in Figure 6, our observed values of F are as follows: 

par 


par* 


Type I: 
Type II: 
Type III: 
Type IV: 


2.0 % VAS 
3.6 % VAS 
5.3% VAS 
12.4 % VAS 


As more years of margin and failure cost data are collected, it will become less 
important to group divisions by profile type. 

(SUMMARY) 

The observed correlation between failure costs and profit performance has many 
uses as a tool of quality management. A particular division which shows no margin- 
to-failure correlation is likely to be reporting failure costs which are not responsive 
indicators of actual performance trends. A division who data does fit a profile-type 
has most need for quality program improvement if its status is "yellow" or "red". 

Perhaps of greatest interest is the slope of the M:F regression line; every dollar 
of improvement in reported failure costs is likely to result in several dollars improve- 
ment in profits. In many instances, this kind of payoff is more fruitful than the more 
traditional efforts to reduce direct product labor or material costs. 



In a nutshell, Shewhart (4) said It first and best: "... when assignable causes 
of variation in quality have been eliminated . . . the product may then be considered 
to be controlled . . . this state of control appears to be, in general, a kind of limit 
to which we may expect to go economically in finding and removing causes of 
variability ..." (emphasis added.) 
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GROUP PROCESSES IN PROBLEM SOLVING 


Hubert M. Hill 
Consultant 

Tennessee Eastman Company 
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Successful problem solving requires the use of strategies, techniques and 
innovative ideas. When working in groups, the problems of the interaction among 
the group members also becomes important. Frequently, due to inexperience or a 
lack of understanding about group processes, interaction among group members reduces 
the effectiveness of the group in solving the problems. 

The purpose of this paper is to describe some concepts and techniques which 
are useful when task forces and committees are solving problems. The specific 
items to be discussed are Cog's Ladder, a model that describes the steps which a 
group progresses through as it matures; and nominal group techniques (NOT) which 
can be used during the problem solving process to reduce the impact of problems 
related to interaction within the group. 

Cog’s Ladder 

Charrier^ has described a model of group growth which he designated as Cog's 
Ladder. This model describes the phases through which a group passes as it matures 
to become an effective team. The model is useful in relating observations about a 
group as problem solving proceeds. 

Cog's Ladder is a five stage model. The stages have been designated as 

(1) polite, (2) why are we here?, (3) bid for power, (4) cooperative, and (5) esprit. 

In the polite stage the group members are becoming acquainted. Controversy is 
avoided, ideas are simple, small talk is the usual course of events. This phase is 
necessary for people to establish some common ground and to begin to feel comfortable 

with each other. It occurs every time the group convenes, even after a number of 

sessions working as a team. 

The second stage, "why are we here?," is used to establish the goals and 
objectives to be achieved. In a problem solving situation, this is when the problem 
is defined and the purpose to be achieved is established. Participation is very 
active on the part of all members and disagreement can occur. T.n this phase it is 
important to build some measure of commitment to the purpose to be achieved. 

The third stage, bid for power, is characterized by competition. The purpose 
of the group has been determined and active problem solving has started. It is 
typical of group members in this phase to propose and lobby for their favorite 
solutions. Group members tend to talk rather than listen and unconventional solutions 
will fall flat. If a group member does not really agree with the purpose to be 
achieved, this disagreement will reappear in this stage. The leader of the group 
can lose power and leadership at this point or he may dominate the group himself. 

When a group moves from the third stage to the fourth or cooperative stage an 
attitude change occurs. Group members actively listen to each other, and are 
willing to change based on what they hear. Creative suggestions are now accepted 
and discussed. Group commitment to the purpose is high as is tolerance of individual 
differences . 

In the esprit stage, cooperation and creativity reach a synergistic level. 

Each member can express his individuality in problem solving. Members of the group 
can disagree while still respecting each other’s views. 
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Moving from stage one to stage two requires that people must accept the possi- 
bility of working on a threatening problem. This may occur if the problem is 
related to a system in which the team member has a vested interest. In moving from 
stage two to stage three, a member may be required to commit himself to working at 
a purpose level which he does not agree with. This can occur in problem solving 
when some team members insist on solving only the immediate problem while others 
believe that considering a broader problem may yield a better solution. To grow 
from stage three to stage four each individual must give up the comfort of defending 
his own views and risk the possibility of being wrong. To grow from stage four to 
stage five demands mutual trust among the team members. 

Problem solving can occur in stages three, four, and five. However, the 
quality of the solution and the effectiveness of the group is likely to be better 
if the group is operating in stages four or five. One of the jobs of the leader is 
to move the group to stage four as rapidly as possible. How rapidly a group moves 
through the stages is affected by the skill of the leader and the techniques used. 

Groups in problem solving usually operate in an interactive mode. Although 
the interactive mode is useful, it has certain characteristics which tend to keep 

the group in phase three. Van de Ven and Delbecq^) have commented on those char- 
acteristics of interacting groups which inhibit problem solving. Those which 
resemble phase three characteristics are (1) self-weighting effect (participation 
in the group only to the extent that the individual feels equally competent with 
others), (2) covert judgements which are not expressed as overt criticisms, (3) the 
tendency of low status individuals to go along with the opinion of high status 
participants, (4) group pressures for conformity and the implied threat of sanctions, 
and (5) the influence of dominant personality types. 

Other inhibiting influences are (1) falling into a rut and following a single 
train of thought, (2) mixing solution generation and analysis, and (3) the amount 
of effort required to maintain the group interaction. 

Nominal Grou p Techniques 

Another group process which can be used in problem solving is the nominal 
group. A nominal group is a group of people working together on the same problem 
but without verbal interaction. 

The nominal group technique (NGT) as developed by Delbecq and Van de Ven is a 
combination of nominal group processes and interactive processes. Using NGT, 
solutions to a problem are generated in the nominal group mode. The participants 
work in silence and record their ideas on paper. When this phase is complete, the 
ideas are solicited from the participants in a round robin fashion and recorded on 
a flip chart pad. When all ideas have been recorded, they are discussed for clari- 
fication in an interactive mode. If a judgement or group decision is required, 
this is achieved by using a nominal process to record the first judgement of each 
participant. These are then listed and discussed interactively. A final group 
judgement is obtained in the nominal mode. Specific instructions for using NGT have 

been described by Delbecq, Van de Ven, and Gustafson . 

There are several aspects of NGT which eliminate problems that can occur in 
group problem solving. Solution generation, clarification and evaluation are 

separated. The problem solving strategy used also can facilitate this separation^) . 

Working in a nominal group forces each person to participate in the generation 
of ideas. By soliciting one idea in turn from each person, each is given the 
opportunity to participate and present ideas. During clarification the discussion 
is directed so as to focus on understanding the idea. Usually, this can be done by 
resorting to the rules of NGT. By making the evaluation and selection of solutions 
in a nominal mode, it is possible to eliminate pressure from higher status individuals. 
If the final step is done in a manner which eliminates the identity of final 
individual decisions, all pressure is removed. 




Whether to use a nominal or interacting mode is a matter of judgement. The 
decision will be affected by the size of the group, the leader's knowledge of the 
group members, his observations on their ability to interact and the task to be 
accomplished. The nominal mode is useful for exploring the facets of a problem, 
generating solutions and entering individual judgements. The interacting mode is 
useful when clarifying or analyzing possible solutions. Large groups can be split 
into smaller groups as described in (3). 

Teams consisting of technical people will tend to move to stage four. However, 
there usually are some members who have difficulty progressing past stage three. 

In our classes in problem solving we have observed that most teams move rather 
rapidly to stage four. On a few occasions a team has actually moved to stage five 
on their last problem. These teams recognized that they were in stage five and the 
quality and quantity of their work was considerably above that produced by other 
teams on the same problem. 

Knowledge of Cog’s Ladder and NGT has been useful to our problem solving 
teams. People recognize situations in terms of the model and can understand what 
is happening. As people become experienced in using NGT, they recognize situations 
where it can be used and apply it. 
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DATA MANAGEMENT - AN APPROACH TO QUALITY CONTROL 


D. H. Williamson 

Manufacturing Engineering Supervisor 
Instrument Products Division 
E. I. DuPont de Nemours & Co., Inc. 
Glasgow, Delaware 19702 


As a complex instrument business begins to rapidly expand, it be- 
comes extremely difficult and ineffective to maintain Quality functions 
by mere manual techniques. The continual evolvement of a new and 
sophisticated diagnostic instrument composed of many discrete subassem- 
blies poses configuration control problems that can readily become 
unmanageable. In order to cope with such a rapidly expanding instrument 
business in a regulated industry, a computerized approach to the analy- 
sis of Quality data was found to be essential. 

This paper describes a Quality Control Data Management System that 
was designed, developed and implemented to provide a means of accumulat- 
ing, storing, tabulating and reporting of Quality parameters such as 
conf iguration control, subassembly location, failure rates, failure 
modes and historical data of instruments and subassemblies. 

The tracking system provides a comprehensive defect and applica- 
tions history of each subassembly from its time of manufacture through 
test, acceptance, depot storage and installation in instruments through- 
out the world. 

Considerations and relative cost trade-offs are discussed in^the 
selection of the type of format and computer system selected (i.e., 
timesharing, dedicated or central processor) . Some actual examples are 
given to demonstrate the effectiveness of computerized data reporting 
in identifying and relating otherwise obscure or undetectable Quality 
problems . 


LCS 346 : 70:428 


1978 American Society for Quality Control 


THE p CONTROL CHART UNDER INSPECTION ERROR 


Kenneth E. Case 
Associate Professor 

School of Industrial Engineering and Management 
Oklahoma State University 
Stillwater, Oklahoma 74074 


ABSTRACT 

One true test of management is the degree to which it is committed to producing 
quality products and services. Although quality cannot be "inspected into" a product, 
the inspection task is crucial in helping identify when an excessive number of 
defectives is being produced. 

The p chart is a classic, well accepted control chart used to determine whether 
or not a state of statistical control exists. Basically, when the observed fraction 
defective falls outside control limits, an assignable cause is sought. Of course, it 
is always implicitly assumed that inspection results are perfect. 

Many independent researchers and practitioners have conclusively shown that 
inspectors do make mistakes, classifying bad items as good and vice versa. Further, 
inspection error rates of as high as 401 are not uncommon. 

This paper mathematically and graphically examines the effects of inspection 
error on the p chart. The point of view where control limits are based upon a target 
value, and the point of view where control limits are based upon error-prone data, are 
both considered. Further, a compensating p chart is offered to help provide an OC 
curve more near to that desired. Also, the OC curve of a p chart under variable error 
rates is explained and illustrated. 

The results of this paper should be of interest to concerned managers who desire 
to turn out a quality product but who may have never considered the possibility that 
their inspection process is imperfect, let alone the ill effects of inspection error. 
The paper closes with a brief discussion of some of the causes and cures of inspection 
error, including a large number of references. All results are illustrated by example. 

INTRODUCTION 

Control charts are an important part of the body of literature known as 
"statistical quality control." The "p" chart, in particular, is a well known attri- 
butes control chart which is easily used, and for which data are often readily avail- 
able. The accuracy of the inspection process in which items are classed as either 
good or defective is fundamental to the use of a p chart. Although there is almost 
always implicit faith in the inspection process, inspection results which cannot be 
trusted cast doubt upon the use of a p chart to indicate the state of process control. 

The entire body of literature on the p chart, including numerous texts which have 
excellent coverage, assumes inspection to be free of error. Conclusive evidence 
exists, however, to show that attributes inspection tasks are far from error- free 
[2, 5, 6, 7, 9, 10, 11, 13, 14, 16, 17, 19]. Even under ideal inspection conditions, 
error rates exceeding 251 are not uncommon [2, 5, 6, 7, 9, 16]. As a result, some 
organizations are taking a hard look at the effectiveness of their inspection proce- 
dures and equipment; however, inspection error will certainly not be eliminated. 

The effects of inspection error have been considered by several authors on various 
aspects of attributes acceptance sampling plans [1, 2, 3, 4, 8, 12, 13, 14, 18, 20, 21, 
22, 23, 24, 25]. In these papers, performance measures such as probability of accep- 
tance, average outgoing quality, average total inspection, total cost per lot, and 
others have been extensively treated. A broad-based introductory paper to attributes 
acceptance sampling under inspection error is given by Collins, et al [4]. Even vari- 
ables acceptance sampling under measurement error is currently generating much interest 
and a growing body of literature. 
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Control charts under error, to the contrary, have received almost no attention, 
with the exception of a recent paper by Abraham [21] covering X, R, and cusum variables 
charts under the assumption of imperfect inspection. There has been no similar paper 
published for the p chart, or for any attributes control chart. 

The purpose of this paper is, therefore, to determine and illustrate the effects 
of inspection error on the operating characteristic curve of the p chart under two 
different (and realistic) points of. view. In addition, a procedure for determining 
control limits which partially compensate for inspection error ivill be developed and 
illustrated. 


NOTATION 


Following is the notation to be used throughout this paper. Summarizing it here 
will facilitate further development. 


n = size of sample taken or considered for p chart purposes 
x = actual number of defectives in the sample 
p = true process fraction defective 

CLp = center line of the p chart based upon the actual or target fraction defective 

UCL = upper control limit of the p chart based upon the actual or target fraction 
P defective 


LCL 


P 


lower control limit of the p chart based upon the actual or target fraction 
defective 


x 


l 


P 

a 

e l 

e 2 

^e 

?e 

PI 

y e u 


7 esl 

P ael 

P2 

C, 'P e 

U Se 

LCL Pe 


es2 


largest number of actual defectives observed in a sample of size n which 
yields a sample fraction defective within control limits 

smallest number of actual defectives observed in a sample of size n which 
yields a sample fraction defective within control limits 

probability that the actual sample fraction defective — falls within control 
.limits n 

probability that a good item will be erroneously classified as defective 
probability that a defective item will be erroneously classified as good 

the number of apparent defectives in the sample. Note that some of these 
may have been incorrectly classified. 

apparent process fraction defective 

a point of view in which the p chart is based upon a target value 

largest number of apparent defectives observed in a sample of size n which 
yields a sample apparent fraction defective within control limits based 
upon the actual or target fraction defective 

smallest number of apparent defectives observed in a sample of size n which 
yields a sample apparent fraction defective within control limits based upon 
the actual or target fraction defective 

probability that the apparent fraction defective — falls within control 
limits based on the actual or target fraction defective 

a point of v.ie\\r in which the p chart is based upon data 

center line of the p chart based upon the mean apparent fraction defective 

upper control limit of the p chart based upon the mean apparent fraction 
defective 

lower control limit of the p chart based upon the mean apparent fraction 
defective 

largest number of apparent defectives observed in a sample of size n which 
yields a sample apparent fraction defective within control limits based upon 
the mean apparent fraction defective 

smallest number of apparent defectives observed in a sample of size n which 
yields a sample apparent fraction defective within control limits based upon 
the mean apparent fraction defective 




CL Pc 
% 
LCL P 
y e i c 


probability that the apparent fraction defective — - falls within control 
limits based upon the mean apparent fraction defective 

center line of the inspection error compensating p chart 
upper control limit of the inspection error compensating p chart 
lower control limit of the inspection error compensating p chart 

largest number of apparent defectives observed in a sample of size n which 
yields a sample apparent fraction defective within the compensating control 
limits 


y esc = smallest number of apparent defectives observed in a sample of size n which 
yields a sample apparent fraction defective within the compensating control 
1 units 

y 0 

P „ = probability that the apparent fraction defective — falls within the compen- 
aec sating control limits n 


DEVELOPMENT - NO INSPECTION ERROR 


The standard Shewhart p control chart has a center line, an upper control limit, 
and a lower control limit. A statistic representing the sample fraction defective and 
based upon the data collected is compared against the control limits. If the statis- 
tic falls within the limits, the user continues to believe that a state of statistical 
control exists. If outside the limits, the user concludes that excessive variability 
exists and that an assignable cause must be affecting the process. 

In this paper, two points of view will be considered. They are as follows: 

PI. The p chart is established based upon a desired fraction defective. In 
this case, no data are used in setting up the chart. Rather, the target 
value, p, is stated as the center line and control limits are developed 
accordingly. 

P2. The p chart is established based upon data. In this case, the center line 
of the chart will be the average fraction defective over an appropriate 
number of observed samples. Control limits will also be determined based 
upon the average fraction defective. 

If "p" is considered the target value in PI or the average sample fraction 
defective in P2, the p chart may be characterized as follows: 


Center Line = 

CL p = P 

Limit = UCL = p + 3 

P 


(1) 

Upper Control 

jEm 

(2) 

Lower Control 

Limit = LCL p = p - 3 


(3) 


When a sample of size n is taken, the sample fraction defective given by the 
statistic — is compared to the control limits. The largest and smallest values of x 
for which an in-control state is indicated are. 


x t = [n-UCL p ] ' (4) 

x s = [n-LCL p ] + (5) 

where [ ] and [ ] + indicate round-down and round-up operations, respectively, to the 
nearest integer. The Operating Characteristic (OC) curve illustrates the probability 
P that a sample fraction defective — will fall within control limits as a function of 
tne true process fraction defective p. That is, 


P = E ( n ) 

c x=max(0,x s ) W 


P X (1-P) n ' x 


0 < p < 1 


( 6 ) 


Example 1 : Consider a process either targeted (PI) or currently operating in 
control (P2) at process fraction defective p = .10. Samples of size n = 100 
are taken periodically. It is desired to describe the control chart, and illus- 
trate its OC curve. 


The control chart is characterized as follows: 


dp ' -10 


UCL 

P 

■ - 10 + 3 { 

,10(.90) = 
100 

.19 

LCLp 

= .10 - 3 ^ 

• 10(.90) = 
100 

.01 

The values of x^ 

and x^ are 


x l = 

[100 • .19]" 

= 19 


x s = 

[100 • .01] + 

= 1 



The OC curve is then expressed mathematically as 


P = 



(l_p) 10 °-x 


0 < p < 1 


This OC curve is illustrated in Figure 1 for a range of p from .005 to .300. 

Note that the OC curve increases very rapidly as small values of p increase, 
levels off at approximately 1.0 in a wide neighborhood about p = .10, and 
decreases very rapidly as p continues to increase. 

— See Figure 1 — 

DEVELOPMENT - INSPECTION ERROR 

The Nature of Inspection Error 

Two types of errors are possible in attribute sampling. An item which is good 
may be classified as defective (type I error), or a defective item may be classified 
as good (type II error) . 

Let 

= the event that a good item is classified as a defective, 

= the event that a defective item is classified good, 

A = the event that an item is defective, 
and 

B = the event that an item is classified as a defective. 

Then, 

P(B) = P(A)P(E 2 ) + PCA)P(E 1 ) . (7) 

By defining the quantities 

P = P(A), true fraction defective, 

P e = P(B), apparent fraction defective, 
e l = the Probability the E^ occurs, 

and 

e 2 = P(E 2 ), the probability that F^ occurs, 

the expression for the apparent fraction defective may be more meaningfully expressed 
as [1, 2, 3, 4, 12, 13, 14, 15, 18, 20, 22, 23, 24, 25]. 

P e = P tl_e 2^ + ( 1- P) e l ( 8 ) 

The use of this apparent fraction defective as observed by the inspector depends upon 
the point of view taken. Both PI and P2 will be considered in the following sections. 

Pl--p As a Tar g et Value 

If p is treated as a target value, the center line and control limits for the 
p chart will be exactly as given in Equations (1) , (2) , and (3) . Now, however, the 



y c 

statistic of interest is the sample apparent fraction defective — where y g is the 
number of apparent defectives observed by the inspector. The largest and smallest 
values of y for which an in-control state is indicated are 

mi - r n - uc y" w 

w - [n-i-cL p ] + . do 

The 0C curve under inspection error is now given by 


P 


ael 


y f 1 / n \ p y e 

y e =max(0,y esl ) \ y e/ e 


d-P e ) 


n-y c 


0 < p e ^ 1 


( 11 ) 


Clearly, although the control chart remains identical to the error- free case, inspec- 
tion error will distort the OC curve of the p chart. 


Example 2 : To illustrate the effect of inspection error let us reconsider 
Example 1 in which the control chart is based on p = .10 and the control limits 
are UCL = .19, LCL = .01. Let the inspection process be confounded by error 
(e,, ejj* . In order 3 to be able to isolate the effect of e^ and e 2 singly as well 

as in combination, perfect inspection plus three error pairs are^considered: 

C e i , e z) = (.00,. 00); (.03,. 00); (.00,. 30); (.03,. 30). The OC curve under inspec- 
tion error is given by 


ael 


, f") 


P e e (l-P e ) 100 ' 


0 < P e < 1 


Figure 2 illustrates the OC curve under each error pair considered. It is 
easily seen that realistic quantities of error have a dramatic effect on the OC 
curve. 


— See Figure 2 — 


From Example 2 it is obvious that an e^ error increases the value of P ae ^ for low 
process fractions defective, and decreases the value of P ael at high process fractions 
defective. Recall that an e^ error is the erroneous classification of a good item as 
defective. Therefore, it is reasonable that errors are made, when the process fraction 

y 0 

defective is low, which increase the likelihood that the sample statistic— falls 
above the lower control limit, causing to increase. Similar reasoning explains 
the propensity for an e^ error to cause the sample statistic to fall above the upper 
control limit, causing to decrease. 


It is also obvious that an e 2 error decreases the value of P ae ^ for low fractions 
defective, and increases the value of P ael at high process fractions defective. Recall 
that an e 2 error is the erroneous classification of a defective item as good. There- 
fore, it is reasonable that errors are made, when the process^fraction defective is 
low, which increase the likelihood that the sample statistic falls below the lower 
control limit, causing P ae ^ to decrease. Similar reasoning explains the propensity 
for an e^ error to cause the sample statistic to fall below the upper control limit, 
causing P ae ^ to increase. 

When both the e^ and e 2 errors are operative together, the e^ error has more in- 
fluence on the OC curve for low p, while the e 2 error dominates the OC curve for high 
p. The reason is that when the actual process fraction defective is quite low, there 
is little opportunity for the realization of an e 2 error. However, as p increases, 
the e 2 error becomes increasingly dominant. These results are consistent with those 
found for attributes acceptance sampling OC curves [4, 12, 22, 23, 24], 


P2-- Control Chart Based Upon Data 

If the control chart is based upon data collected under inspection error, its 
center line and control limits will be modified from Equations (1), (2), and (3) . 
Specifically, the center line will now be the average of a large number (often about 

e 


30) of values of the sample statistic. — . 
this value since it is the mean of this sample statistic, 
on this value results in 


For simplicity, the notation p g is used for 
Basing the control chart up- 


Center Line = 

CL = p 
n 

1 e 


(12) 

Upper Control 

Limit = UCL = p + 3 1 

P e e J 

l p e a-p e ) 

(13) 


c V 

n 


Lower Control 

Limit = LCL = p 0 - 3 

* P + 

1 Pe (1 "Pe^ 

(14) 


Just as in PI, the statistic of interest is the sample apparent fraction defective — . 
The largest and smallest values of y & for which an in-control state is indicated 
will now be 


es2 


=rn*UCL 1 ' 

(15) 

L P eJ 

= fn • LCL 1 + 

(16) 

L P eJ 


The OC curve in this case is given by 


ae2 


y el2 
l 

y e =max(0,y es2 J 


(r t 


a-p e ) 


n-y r 


P < 1 
*e — 


(17) 


Now the control limits as well as the sample statistic are based upon error-prone in- 
spection; hoxvever, it is unclear from Equation (17) what effect this will have on the 
OC curve. 


Example 3 : Again let us consider Example 1, and assume that the p chart is now 
based upon data observed from an error prone inspection process operating at an 
actual fraction defective p = .10. The same four fractions defective considered 
in Example 2 will be used: (e^^) = (.00,. 00); (.03,. 00); (.00,. 30); (.03,. 30). 

The center line and control limits will change as a function of each error pair 
considered. 


To illustrate the calculations underlying the p chart in this case let 
(e r e 2 ) - (.03,. 30). 

Center Line = CL = p £ = PCI" 1 ^) + (l-p)^ = (.10)(l-.30) + (1-.10)(.03) = 


097 


Upper Control Limit = UCL 


097 


+ 3 V 


097(1-. 097) = .18579 
100 


Lower Control Limit = LOL = .097 - 3 J .097(1- .097) = .00821 


{ 


100 


The values of y e ^2 an< ^ ^es2 are y el2 ~ * *18579 J = 


100 • .00821 


^es2 L 

The OC curve under this error pair would be given by 

100-y o 


r- 


ae2 


J /100\ 

i Iw 


a-p e ) 


0 ip e <i 



Figure 3 illustrates the OC curve under each error pair considered. Table I 
presents the center line and control limit values relevant to this example. 

— See Figure 3 — 

Table I. Center Line and Control Limit Values 


for Example 3. 


(e l> e 2) 

cl p 

1 e 

UCL 

p e 

LCL 

*e 

(.00, .00) 

.10000 

.19000 

.01000 

(.03, .00) 

.12700 

.22689 

.02711 

(.00, .30) 

.07000 

.14654 

.00000 

(.03, .30) 

.09700 

.18579 

.00821 


In Example 3, each of the OC curves under inspection error deviates from the de- 
sired OC curve marked by ( e pe 2 ) = (.00,. 00). For high values of p these deviations 

are not nearly as severe as those from Example 2 illustrated in Figure 2. This is be- 
cause the control limits used in Example 3 are actually based upon the error prone 
data, and partially compensate for the variability of the sample apparent fraction de- 
fective. For low values of p, the deviations from the desired OC curve are somewhat 
extreme. This is largely due to the particular example illustrated, having a lower 
control limit so near .00. 


COMPENSATING p CHART 


A center line and control limits may be determined for the p chart which, to a 
large extent, compensate for inspection error and provide an OC curve closer to that 
desired. The basic approach to be presented is applicable to both PI and P2. 

First the center line and control limits are determined using Equations (1) , (2) , 
and (3) . This is easily done when point of view PI is employed, based upon the target 
value or actual fraction defective p. However, approach P2 relies upon the apparent 
fraction defective p 0 . Thus, the value of p to use in Equations (1), (2), and (3) may 
be found by solving e Equation (8) for p: 

.P = P e " e-L (17) 

Once the center line CL p and control limits UCL p and LCLp have been determined, 
the compensating center line^O, and control limits UCL„ and LCL„ are calculated as 
follows: p c p c p c 


Center Line = CL p = CL p (l-e 2 ) + (l-CL p ) 


(18) 

Upper Control Limit = UCL = UCL (l-e ? ) + 
P c p * 

(l-UCL p ) ej 

(19) 

Lower Control Limit = LCL = LCL (l-e 2 ) + 

(1-LCLp) ej 

(20) 


Note that the compensating control chart simply reflects taking the error-free center 
line and control limit fractions defective, and using Equation (8) to convert them to 
error-prone equivalents . 


The largest and smallest values of y for which an in- control state is indicated 
are 


' - = f n*UCL 1 ‘ 

elc L p cJ 

(21) 

r = T n • LCL 1 + . 

esc [ P C J 

(22) 


The OC curve using the compensating control chart under inspection error is now given 
by 


y e2c 

2 


y =max(0,y ) 


a-p e ) 


n-y c 


0 < p < 1 


(23) 



Example 4 : Continuing with the same basic example, let us determine compensating 
control charts and OC curves for each error pair of interest. For illustration 
purposes, only the control chart for (e, ,e 2 ) = (.03,. 30) will be considered. It: 
is already known from Example 1 that Cl/ = .10, UCL = .19, and LCL = .01. Using 
Equations (18), (19), and (20): P P P 

CL = .10(1-. 30) + (1-.10) .03 = .09700 

UCLp = .19(1 -.30) + (1- .19) .03 = .15730 
*C 

LCL = .01(1-. 30) + (1-.01) .03 = .03670 
p c 

The values of y 7 and y are y 7 = |100 • . 15730 ] " = 15 

7 e7c 7 esc ' elc j 

y esc ■ [lOO • .03670] + = 4 

The OC curve under this error pair would be given by 


P 

aec 





U-p e ) 


100-y e 


0 < P e < 1 


Figure 4 illustrates the OC curve under each error pair considered. Table II pre- 
sents the compensating center line and control limit values relevant to this 
example. 


—See Figure 4 — 


Table II. Compensating Center Line and Control 


Limit Values for Example 4 


( e l, e 2 ) 

a P 

1 c 

ucl p c 

La 

1 C 

(.00, .00) 

.10000 

.19000 

.01000 

(.03,. 00) 

.12700 

.21430 

.03970 

(.00, .30) 

.07000 

.13300 

.0p700 

(.03,. 30) 

.09700 

.15730 

.03670 


In Example 4, the compensating control charts for each error pair provide OC 
curves which are quite close to the desired curve at (e,,e 2 ) = (.00,. 00). In fact, 
the method presented for designing compensating control "charts provides exactly the 
desired OC curve as n increases without bound. Practically speaking, this means that 
the compensating control chart provides better results for larger sample sizes. 

NON- CONSTANT INSPECTION ERROR 

Biegel [2] proposed that for some inspection processes, inspection error is a 
linear function of the actual fraction defective p. This is in contrast to the con- 
stant error rates used to this point. Specifically, he assumes Types 1 and 2 error are, 
respectively, given by 


e 1 (p) = l-ta-j+bjp) 

(24) 

e 2 (p) = l-(a 2 +b 2 p). 

(25) 


With the exception of these changes, the equations relevant to points of view PI and 
P2 remain the same. 

Example 5 : As a matter of interest, consider the same basic problem faced in 
Example 2, but now allowing error to vary linearly with the actual fraction de- 
fective. Using results very close to those of Biegel [2] based on data 

e x (p) = 1 - (l-.1518p) and e 2 (p) = 1 - (.6884+.4190p) 

Using a p chart having CLp = .10, UCLp = .19, and LCLp = .01, the resulting OC 
curve is contrasted with the error- free OC curve as shown in Figure 5. Table III 
lists the error pairs and the apparent fraction defective p g for a few values of 
the actual fraction defective p. 



-- See Figure 5 -- 


Table III. Selected Values of p, e^Cp), e 2 (p), and 

P e Used in Example 5 

P 

e 1 (p) 

i ^ 

<u 1 

i 1 

Pe 

.05 

.00759 

.29065 

.04268 

.10 

.01518 

.26970 

.08669 

.15 

.02277 

.24875 

.13204 

.20 

.03036 

.22780 

.17873 

.25 

.03795 

.20685 

.22675 


SUMMARY AND CONCLUSIONS 

This paper has shown that inspection error rates which are not unrealistic in 
industry seriously affect the OC curve of a p control chart. It may come as a surprise 
to many that inspection error exists at the levels illustrated, and that it can distort 
the OC curve so badly. To those who have been concerned with inspection accuracy prob- 
lems, the results illustrated in this paper should come as no real surprise. 

One of the features of this paper is a method of developing a control chart which 
partially compensates for inspection error. However, the author recommends that such 
a compensating chart be used only as a last resort after serious efforts have been made 
to minimize inspection error. Areas where inspection improvement efforts are likely to 
pay off include inspection station design, improved communications including feedback, 
tool and technique improvement, and inspector selection. Even more than these, in- 
spector training or retraining will almost always be worthwhile- -especially when judg- 
ment is an integral part of the inspection task. A superior treatment of causes and 
cures for inspection error is presented in [16] . Also, an excellent treatment of human 
inspection performance is available in [5] . 
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Figure 5: Example 5 OC Curve (Using Biegel [2] Variable 
Error Rates) 
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GEOMETRIC MOVING AVERAGE CHARTS AND TIME SERIES 
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and 
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1. INTRODUCTION 

Roberts (5) introduced some control charts based on geometric moving averages 
(GMA) to add sensitivity in detecting small shifts in a process average. Here we dis- 
cuss an interesting connection between the GMA and time series forecasting. We also 
introduce some modified charts based on weighted averages and discuss their use in 
checking forecast stability. 

In Section 2 we describe the GMA briefly as given in Roberts (5) and Bather (3). 
Section 3 shows (i) GMA as a one step ahead forecast and (ii) a Control Chart for 
checking forecast stability. Some generalisations of the results in Section 3 are 
given in Section 4. 


2. THE GEOMETRIC MOVING AVERAGE CHARTS 

We suppose that a process is on target Uq (=0, for convenience) initially and 
successive measurements Z ^ (t = 1,2,...) are taken (it could even be average of 
several measurements taken at time t) to check whether there is a shift from the tar- 
get. Then, following a suggestion from. J.W. Tukey, Roberts (5) proposes to use a con- 
trol chart based on the statistic 

g t = r[Z t + (1 - r)Z t _ 1 + ... + (1 - r)*"^], t > 0 (2.1) 

which is a geometric mean of all the observations ^,...,Z^ with Z^ the most 

recent observation getting the greatest weight (=r) and all previous observations 
weights decreasing in geometric progression. Assuming that Z is drawn independently 
from a normal distribution with variance a 2 and that t is sufficiently large so 
that a 2 = [r/(2-r)]o 2 , one could form a control chart quite similar to the ordinary 
Shewart®Chart . One of the disturbing things here is that r is quite arbitrary and 
Roberts (5) takes r = 2/5 because "it may have wide appeal in practice" (note when 
r = 2/5 , a 2 = a 2 / 2 ) . 

Bather (3) arrives at the statistic given in (2.1) through a Bayesian argument. 

We outline his procedure below. Suppose that a machine whose performance can be ef- 
fectively represented by a single unknown quantity p_ t , is inspected regularly to see 
whether the quality of performance is deteriorated. The successive performance levels 
p.^,p,„, * " * ’ !' l t are trac ^ e< ^ by the observations Z_,Z_,...,Z . The machine is assumed 
to start from a repaired state, i.e. p 0 has a desired (target) value (=0, for con- 
venience) . The operation continues until a decision is made to overhaul it in which 
case the level is set to zero instantaneously and the whole sequence begins again. 

This resetting after overhaul may be subject to error and so it is assumed that p.g is 
N(0,Vg) (normally distributed with mean zero and variance v^) and each subsequent 
state of repair is drawn independently from this distribution. Now assuming that 
P t - are drawn independently from N(0,a^) and that Z is N(p_ t ,a 2 ) Bather 

(3) shows that the posterior distribution of p, given Z t ,Z ^,...,Z is N(u ,v ) 
where the posterior mean u = E(u |Z ,Z . , . . . , Z. ) is sufficient for 1 Z , ...,Z.T. C He 
also shows that 1 

u t = (l-q)[Z t + qZ t< _ 1 + ... + q t " 1 Z 1 ] (2.2) 

where q is a function of a 2 and crp. 2 . The statistic g fc and u t are the same 
when r = 1-q. Now successive decisions can be made by means of a control chart on 
which the values u^,U 2 S ...,u t are plotted until a point falls inside a stopping 
region indicating that an overhaul is needed. 

it 
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3. GMA AND FORECAST STABILITY 


As before we suppose that observations . . . , Z t , . . . are made at regular 

intervals and the underlying structure can be represented by a time series model 

Z t - Z fc _ 1 = a t - 0a t _ 1 , jel <1, t = ...-1,0,1 ... (3.1) 

where {a^} is a sequence of independent identically distributed random variables with 
N(O,0 2 ). When 0=0 this is the so called random walk model. We consider (3.1) here 
because it has been used to represent a wide variety of economic and industrial time 
series . 


The model (3.1) can be re-written as (see for example Abraham and Box (1)) 

Z 


where 


't+1 

(1 - 0) [7, + QZ 


h + \+i 


t-1 


+ ... 


vt-1. 


.] 


(3.2) 

(3.3) 


and when t is sufficiently large f t is the same as u with q = 0. Now the min- 
imum mean square error (mmse) forecast of from t is the conditional expec- 
tation of given Z^jZ^ ( see Box a hd Jenkins (4)). This is given by 




(3.4) 


which is an exponentially weighted average (EWA) or GMA of Z ,Z ^,... . For com- 

putational purposes f^ can also be written as 

f t - 9 f t _! + (1 - 0)Z t (3.5) 


implying that the computation can be done recursively as a new observation becomes 
available. Here one can obtain estimates (9,0 2 ) of (9,0 2 ) using time series esti- 
mation techniques (see Box and Jenkins (4)) and this eliminates the arbitrariness of 
r = (1 - 0) which Roberts (5) had to cope with. 


Since f also represents the level of the series at time t one could plot it 
successively on a control chart to see whether there is a drift in the level. Here, 
however, we will consider another important problem namely the checking of forecast 
stability. One may be using f t as the one step ahead forecast when in fact the para- 
meter 9 in the model (3.1) has changed or the model (3.1) itself may have changed at 
some point in time. This could lead to (i) forecasts which don't have minimum mean 
square error and (ii) forecast errors which are autocorrelated. We refer to them as 
forecast unstability. 


Suppose that the model remains the same. Then a c+ j = Z ^ - f t , the one step 
ahead forecast errors are drawn independently from N(0,O 2 ). Hence one could monitor 
these a t ' s to see whether or not the model is changing. 


If there is a parameter change at (t + 1) the model may be written as 

z t+i - z t = a t + i - (e + K)a t (3 - 6) 

where K is the amount by which 0 is changing. Now (3.6) can be re-written as 

Z c+1 «f t+ A t (3.7) 


where = d t + a t+ ]_ and d^ depends on 0,K,Z ,2^ ^,... . In this case E(A t ) ^ 0 

and A t 's are autocorrelated. Hence we propose to use the following simple Shewart 
Chart to detect forecast stability. 

Step 1. Obtain (0,0^) and some action limits based on a (probably 20 units 
away on either side of 0). 

Step 2. Plot - f^ successively until a point falls outside the action limits 

or some 'autocorrelation among the a t ts is noticeable. 

Step 3. Modify the model to get the minimum mean square error forecast. 


Construction of a Shewart Chart is proposed here because of its simplicity. How- 
ever when more sensitivity is required (which is often the case in forecasting) one 
should probably use a cusum chart (see Barnard (2)) based on the forecast errors a fc . 



4. A MORE GENERAL MODEL, WA AND ^ CHARTS 

Now we generalise the structure given in (3.1) and consider the observations to 
be generated from an Autoregressive Integrated Moving average model of order (p,d,q) 
defined as 


4>(B)V d Z 


where 4>(B) = 1 


0(B) a 
0(B) = 


t = . 


- 1 , 0 , 1 ,. 


1 - 


(4.1) 
B is a 


backward shift operator defined as BZ = Z._-, and a is^as defined in Section 3. 
It is also assumed that the roots of $(B)9(B) = 0 lie outside the unit circle. The 
model (4.1) can be re-written as _ 

Z 


t+1 


t+1 


X ”i z t+i-i 


i=l 


where the weights w^ satisfy the relation 


e _1 (B)cf)(B) (1 - B) d = w(B) = 1 - - w 2 B 2 - . . 


(4.2) 


(4.3) 


As before the mmse forecast of Z t+ ^ from t is 


E(Z 


t+1 


z , z _,...)■ y Wj 

t* t-i* i 


i=i 


‘t+l-i 


(4.4) 


Also One can show that £ w^ = 1 implying that f is a weighted average. In any 
real situation one has only Z^ , Z fc ^ , . . . ,Z^ and hence one could take f ^ as 

^ w Z , . This approximation is not unreasonable because we expect the weights w. 
i=l t . 1 
to die out fast (the roots of <j>(B)9(B) = 0 lie outside the unit circle). 


The procedure outlined in Section 3 can now be adopted to check forecast stability. 

Step 1. Estimate (<£>&' ,G^) = (<f> , . . . , 4> p , 0 , . . . ,0 , a ^) using Time Series estimation 
techniques (see Box and Jenkins (4)) and^obtain action limits based on a . 
Step 2. Calculate the one step ahead forecast f t of 2^+1 using one of the 

techniques (difference equation approach) given in (4). The formulae (4.4) 
or its approximation is not best suited for successive computation. 

Step 3. Plot the points ~ ^t successively. 

Step 4. When a point falls outside the action limits or when some autocorrelation 
among the a ' s is noticeable, the underlying model needs modification. 

The chart described here will be referred to as a Weighted Average (WA) Chart. The 
forecasting system can be designed, without much difficulty, such that the WA Chart is 
constructed and the points Z t+ ^ - f fc is plotted automatically by a computer, after 
the estimation stage in Stepl. 


Now we indicate the possible use of another chart which we refer to as the First 
Lag Autocorrelation (r^ ) Chart, to monitor whether or not there is autocorrelation 
among the one step ahead forecast errors (see Step 4) . Here we propose to plot 
t t 2 

r.. = I 1 / I a- successively on a control chart to see whether the first lag 

i=2 1 i=] 

autocorrelation is significantly different from zero. Action limits may be placed on 
either side of zero at a distance of 2.S.E(r^) where S.E(r^) = l//t. Ideally one 
should check the first few autocorrelations. However, to keep the complexities to a 
minimum, it would probably be enough for our purposes to monitor r^ alone. 

It should be noted that the r^ chart itself may not be enough in a given 
situation because, in general, there will be some lag between the times at which the 
change actually happened and at which the change is noticed in the r^ chart. Hence 
we favour the use of the WA and r^ charts simultaneously. 


*when d is equal to or less than one 



5 . CONCLUDING REMARKS 


The charts based on GMA as described in Roberts (5) is more complex than the 
Standard Control Charts and this complexity may not, in many cases, be compensated by 
the added sensitivity. However, if the observations have a structure as in (3.1) the 
mmse one step ahead forecast is a GMA and the new chart based on this to check fore- 
cast stability is relatively simple and very interesting. The more general structure 
in Section 4 leads us to WA and r^ charts which seem to be very promising. 
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VARIABLES ESTIMATES OF PERCENT DEFECTIVE 
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ONE-SIDED SPECIFICATION LIMITS 

We start by considering example D-2, p.90 of MIL-STD 414. "The specified minimum 
yield point for certain steel castings is L = 58,000 psi. The variability a is known 
to be 3,000 psi. The yield points of a sample of size n = 10 are 62,500; 60,500; 
68,000; 59,000; 65,500; 62,000; 61,000; 69,000; 58,000; 64,500." On the basis of this 
sample, we wish to estimate the fraction defective in the lot from which the sample 
was drawn. 


The recommended procedure is to compute the quality index 

(x-L)'l/ n 


Q L a 


n-1 ’ 


where x is the sample mean, and estimate the fraction defective P^ as the probability 
corresponding to the normal deviate z = -Q , i.e., 

P L = F(-Q l ) = F(z) = / Z exp(-t 2 /2)dt/ 2ir 


For the example above 


( 63,000 - 58,000 )1/10 = 1.76 , 
3,000 


P L = .0392 = F(-l. 76) 

A more natural, but naive, approach to the estimation of P^ is to note that if u is 
the lot mean then, 


Thus, since x is an estimate of y 


P L = F 


P L - F 




-x+L 

a 


would be a intuitive estimate of P^. For the example, this gives the estimate 
Pl = f ^ 58 > 00 ^~ - |^ Q ^ = F (-1 . 6667) = .0482 . 

This differs from the recommended estimate 


P L = .0392 


by almost 1% defective. 

The estimate P^ , as derived in reference (l ), is a minimum variance unbiassed 
estimator, and thus "has many good "Jarge sample" qualities. On the other hand, an 
examination of the distribution of indicates that other estimators have properties 
which are better in certain circumstances. 

THE DISTRIBUTION OF P L 

Let X. , X- , . . . X be an independent random sample from a lot which is normally 
distribution with mean y and standard deviation a. Let Z be a N(0,1) variable with 
density function 

f(z) = exp(-z 2 /2)/V , ?ir 
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and distribution function 


F(z) = j Z f(z)dt 


The fractile points z(t) are defined as t = F(z(t)). 

Let L be the lower specification limit for the lot, then 


where 


The estimator is defined as 


which is distributed as 


P L - F(z(P l )) 


Z(P ) - Z(L) - hr 
L a 


P L= F 


/-xf LL / vT ) 

{ a |n-t ) 

= y ( uz(L) - Z ) 

L ' F ( fs=r J 


where Z is N(0,1) . 

The distribution function of P, is then 


G(t) = Prob (P L <t) 


- Prob ~ -) <t) 

- 

= F (fn-l z ( t) -fn z(L)) , 


and the density function is 


f(t) = dG(t)/dt 

= \/n-l f (jln-l z(t) -^tT z(L)) dz(t)/dt 
* f n-T exp(-((n-2)z(t) 2 - ZlTii^rFl z(t)z(L) + nz(L) 2 )/2) 
where, as before, t = F(z(t)). 

From the distribution function we note that F(t) = .5 implies that 

fn-l z(t) -fa z(L) = 0 

z(t) *|f 5 & (L) 


that is. 


median = F 


■ '(| 0 T 


The estimator whose median is P, 




The density function may be represented as 


g(t) = fn-l exp (-(—pKz(t) - ~ z(L) ) 


fiirr i 


2 . _n_ z(L) 

n-2 2 


>) 


from which it is obvious that the mode occurs when 



z(t) 


fnfrFT 

n-2 


:(L) 


f( £MbzI z(l)) 

n-z 


Hence an estimator whose mode is is 

P T = F ( r~ ^ 

L n-i a 


For the example data the three estimators are 

P T = F( — ) = F (-1. 76) = .0392 
L V n-i a 


P T = F( ^ ) = F(-1.67) 
L o 


.0475 


P T = F ( ) = F (-1.48) - .0694 

L n-i 0 

Figure 1 shows the density function of P for samples of size n = 10, 20, 50 from 


10% defective lot (P T 


10). The mean, median and mode for these densities are 



n=10 

n=20 

n=50 

mean 

.10 

.10 

.10 

median 

.088 

.094 

.10 

mode 

.064 

.083 

.093 


TWO-SIDED SPECIFICATION LIMITS 

When both upper and lower specification limits ( U and L ) are given the unbias- 
ed, minimum variance estimator for P, the lot fraction defective is 


P = P, + P - F< ^P-) + F ( 2^/P- ) 
L U a J n-1 a £/n-l 


|/n-l 

This is merely the sum of the estimates of the fraction outside the individual limits, 

For the example data, with upper specification limit U = 67,000 psi (this is ex- 
ample D-3, p.97, MIL-STD 414). The estimates are 


P = p ,63 1 000 = 58 1 000 10 = 

F L 1 3,000 ' F( “ 1 * /b) 


.0392 


P, " ) - F(-1.41) - .0793 

P = P L + P y - .0392 + .0793 = .1185 

It may be noted that since U - L = 3a, the minimum lot fraction defective occurs 
at p=(U+L)/c and equals 

P L + Py = F(-1.5) + F(-1.5) = .1336 . 

The minimum estimate occurs when x = 63,000 and is 

10 


P = 2F( -1.5^ ) 

a seeming contradiction. 

The distribution function of P is given by 


,1164 , 


G(t) = Prob (?<t) = Prob (P L + P y <t) 


= Prob (F((L-x)V/o) + F( (x-U) V/a) <t) 


- Prob (F( f^- Z ) + F( CnzM + Z )<t) 
y n— 1 \f n- 1 




Figure 1. Density functions for P^ 
(P L = 10, n - 10, 20, 50) 



I l 


Figure 2. Density functions for P. 
(P = P L + Py = .10, n = 10) 



where, as before 


F(z(L)) = P L , F (z (U) ) = Py, P = P L + Py 

Z is N(0,1) 

V = Y~n/ (n-l)~. 

Since and P^ are both unbiassed, it automatically follows that 
E (P) - E(P l ) + E (Py) - P L + p u = p 
and hence P is unbiassed. 

The minimum value of P occurs when Z = ^n(z(L) - z(U))/2, hence, 

P = 2F(V(z(L) f z(Uj)/2) 
mm 

TABLE I 

Characteristics of some 10% defective lots 




(P=.10, n=10) 


P L 

p u 

z (L) 

z(U) 

Pmin 

.10 

.00 

-1.281 

— 06 

0.000 

.09 

.01 

-1.341 

-2.326 

.053 

O 

CO 

.02 

-1.405 

-2.054 

.068 

.07 

.03 

-1.476 

-1.881 

.077 

.06 

.04 

-1.555 

-1.751 

.081 

.05 

.05 

-1.645 

-1.645 

.083 

form 

of G(t), it is obvious that 

if Z = 

z(t) is a solut 



z(E) ~ Z . Ff /n 

z(U) + 

_Z x - f 



/n=T } + F( 

y/ff-i 

) ~ t 

!(t) 

+ Ifn (z(L) 

- z(U)) is also a 

solution. 


The density function of P is 


g(t) = dG(t) / dt 


V n-1 f(z(t)) + f(z (t)) 


f(- 


z(t)-l / 'nz(L) 

VTPI 


) - f( 


z(t)^z (IJ) 
l' n-1 


for t between t = P . and t = 1. 

min 


Figure 2 shows the density functions of the lots listed in TABLE I. In each case 
the density function is infinite at P = P . . 


VARIABILITY UNKNOWN 

Where the lot standard deviation a is unknown, the sample standard deviation 


f'L (x-x 
n-1 


72 


must be used. The statistics 


s ’ 


u-x 



are used in place of 


x-Lfn U - x y° 

a I n-1 ’ a f n-1 

the factor |/"rT/ n-1 is included in table B-5, which gives the unbiassed estimates of 
the fraction defective. 


The estimators and are functions of the non-central t-distribution and a 
discussion of their properties will be published elsewhere. One peculiarity which 
might be noted is that if 


Q 


L 


x-L . (n-1 ) 

a y/T 


then P^ = 1 and conversely if 




x-L . n-1 
a YTT 


then P^ = 0. 


MATHEMATICAL NOTES 


The standard attributes estimator for the fraction defective is 

P I* * d/ “ 

where d is the number of defectives (values of x < L) in a random sample of size n. 

The corresponding unbiassed minimum variance attributes estimator as given by Black- 

well (ref. 1) is 

P L = E(P l *| I) 

Where T is sufficient for P , i.e. T = x if o is known and T = (x,s) if a is unknown. 

Derivation of the properties of P , as used in MIL- STD 414, are given in references 

(2) and (6) . The variance of P- xs given by Owen (3) in the case o is known and by 

Wheeler (4) when a is unknown. L The denisty functions given here appear to be new. 
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INTRODUCTION 

Statistical methodology and the QC Circle are inseparable. The QC Circle was 
founded upon the principles of problem solving. Consequently, statistical tools form 
an integral part of Circle activity. The authors first review these various tools and 
then discuss the need for statistical theory in the development of new tools for appli- 
cation by Circles. Finally, a call is made for research into the QC Circle phenomenon 
itself. 


THE QC CIRCLE 

The QC (Quality Control) Circle is a group of individuals whose function is the 
furtherance of quality control. It is a small group of about five to ten members. 
Usually the membership is comprised of people who work together as in one section of an 
assembly line, a machine shop, or department. Host QC Circles operate at the level of 
foreman together with hourly paid personnel; however, some Circles are formed at higher 

levels such as a Circle of engineers in Lockheed Missiles and Space Company Gen- 

erally the leader of a Circle is the foreman or assistant foreman although, as at Honda 

( 2 ) 

Motor Company , there may be a deliberate attempt to encourage operating personnel as 
leaders. The Circles meet on a regular formal basis from one hour per month to as fre- 
quently as one hour per week. An essential ingredient of the QC Circle is the volun- 
tary aspect of membership; no one is forced to join, although the Circles and manage- 
ment may encourage participation. (Even in Japan where one might think that 100% par- 
ticipation occurs, this simply is not true. The author and his wife were given the 
opportunity at Honda Motor Company to sit in on a meeting of foremen who were address- 
ing the question — How may Honda encourage non-participating employees to join QC 

Circles?) 

The QC Circle’s sole objective is to solve work-related problems. The term "QC” 
is technically a misnomer because the Circles will address a variety of problems other 
than those which are commonly considered to be under the domain of quality control. 

The QC Circles in Japan were estimated to spend about 50% of their effort on quality 

control. ^ Reduction in costs and productivity increases together accounted for 40% 
of the effort and there were a variety of other areas of endeavor. It is important to 
see that the Circles concentrate on the solution of technical problems; they never ad- 
dress salary or other issues which lie in the bailiwick of management and organized 
labor. 

It is well worth noting that QC Circles themselves carry through the complete 
problem solving process of: 

a. identifying problems, 

b. deciding what problems to address, 

c. solving that problem, 

d. implementing their solution, and 

e. maintaining the solution. 

It is essential to recognize this problem solving feature in QC Circles in order to 
differentiate this activity from other concepts such as the Product Teams in Sweden 
or the suggestion system. 
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HISTORY OF QC CIRCLES 

The QC Circle originated in Japan in 1961. ^ The entire scope of events which 
led to its beginning is much too broad to fully describe in this paper, yet, a brief 
review may overemphasize some aspects and neglect other essentials. Dr, Ishikawa re- 
marks that the QC Circle is a "logical development of our endeavor of breaking through 

the above stated barriers which have confronted Japanese industry at large. (See 
reference for the list of seven barriers.) In addition, several salient points may be 
made. First, the Circle is seen as a way to enhance the development of the foreman as 
well as the operators (hourly paid employees). Secondly, the Circle provides an op- 
portunity for the foreman together with operators to actively improve their own job by 
means of solving work-related problems. Thirdly, the problem solving nature of Circle 
activity serves as an outlet for a basic human need: creativity. An off-shoot or 
fall-out of this creativity outlet is that the Circle activity gets to the heart of 
motivation. Fourth, the problem solving activity of the Circle increases the profit- 
ability and productivity of the firm. 

Contrary to some opinions, the United States has supplied many of the "building 
blocks" out of which the QC Circle was constructed. Dr. Joseph Juran and Dr. W„ 
Edwards Deming were invited to give a series of lectures and seminars on the manage- 
ment of quality control and statistical methodology, respectively. It is from these 
inputs, Japan's commitment to quality, and the threefold objective of the QC Circle to 

"1. contribute to the improvement and development of the enterprise, 

2. respect humanity and build a worthwhile to live and happy bright 
workshop , 

3. display human capabilities fully and evantually draw out infinite 
possibilities 

that the QC Circle emerged in 1961. A phenomenal training effort took place in Japan: 
radio and TV courses; seminars both within and outside companies; and magazines and 
journals. Tremendous growth in numbers of QC Circles has taken place, until, the num- 
ber of Japanese QC Circles registered at the JUSE (Union of Japanese Scientists & En- 
gineers) was approximately 70,000 in 1975. With about ten members per Circle, this 
means that about 700,000 operators and foremen are actively engaged in solving work- 
related problems. Moreover, estimates have been made that ten times this number are 
really Involved, since possibly only 10% of the Circles are registered. 

From the inception of QC Circles in 1961, the concept has jumped national and 
cultural boundaries. Today Circles are active in South Korea, Taiwan, Singapore, 

Brazil^, Mexico the United States , and in Holland . 


STATISTICS USED BY QC CIRCLES 

The immediate objective of a QC Circle is the solving of work-related problems. 
The framework of a sequence of steps in this problem solving process is shown in 
Figure 1. 

Identify Goals 

The first step is the identification of goals. This thrust may come from manage- 
ment or may arise from within the Circle. When the goal is broadly defined, such as 
the reduction of defects in a particular department, then the need arises to identify 
the component problems. Next, choices must be made as to which components are t:o be 
tackled first. At this stage data collection becomes essential. 

Data Collection 


The Circle must understand several aspects of data collection: the purposes of 
collecting data; is it correct data such that the structure will be revealed; the 

( 12 ) 

kinds of data; organization of data; and how to prepare before collecting data. ' 


The Japanese, in their insistence that the QC Circle carefully and purposefully 
collect data, seem to have grasped a basic principle put forth by the late eminent 
statistician, Sir Ronald A. Fisher. Professor Fisher in his book Design of Experi- 
ments states: "Statistical procedure and experimental design are only two different 


( 13 ) 

aspects of the same whole ..." When the QC Circle collects data, it does so by 

design - by predetermined plan - and the data is collected so that a proper interpreta- 
tion of the facts can be achieved through analysis of the data: this is statistical 
procedure in the fullest sense. 



* IDENTIFY GOALS 

* COLLECT DATA 

* ANALYZE DATA 

- PARETO ANALYSIS 

- FREQUENCY HISTOGRAMS 

* BRAINSTORM CAUSES A 

* EXPERIMENT 

* ANALYZE RESULTS 

* TEST SOLUTIONS 

* IMPLEMENT SOLUTIONS 

* DESIGN & IMPLEMENT SAFEGUARDS 



FIGURE 1. Problem Solving Sequence 


Pareto Analysis 

Pareto analysis is one of the most important techniques used by QC Circles since 
it serves to keep the attention on the most serious, and potentially most rewarding 

Q4) 

problems. ' The four major ingredients of this technique are: 

a. a desire to effect improvement 

b. data collection 

c. summarization of the data in a Pareto Chart (see Figure 2) 

d. follow through on the biggest problem as identified on the 
Pareto Chart. 



FIGURE 2. 


Pareto Diagram 
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Commonly only items (b) and (c) would be considered the technical aspects of Pareto 
analysis; however, the analysis will be viewed as a sterile exercise unless ingredi- 
ents (a) and (d) are included. 

Frequency Histogram 

Frequency histograms are a second analytical tool used by the QC Circle on data. 
The histograms will show distribution type, centering, variation and any abnormalities 
When the specifications are drawn on the histogram, the Circle will be able to see not 
only the process capability but also the ability of the process to meet the specifica- 
tions. (See Figure 3) 



FIGURE 3. Frequency Histogram 


The usefulness to the Circle of the frequency histogram is due to several factors 
simplicity of the histogram; ease of construction; effortless interpretations; and 
power of the tool to show process capability with respect to specifications. 

Check Sheets 


The check sheet is a form for the collection of data. It is designed so that it 
may be easily used and the data "organized automatically." As illustrated in Figure 4 
the Circle may use the check sheet to assure that the data are collected in such a way 
that the check sheet will reveal several features: machine, shift, and day. Then it 
is an easy matter to sum up the data within each of the categories and then examine it 
for major differences. 



Lshiftl 

Mon 

Tue 

Wed 


Fri 

Totals 

Machine A 

m 


d 



■H 

4 

2 

IHI 

df 


| 

■I 

1 

Machine B 

1 

XX 

xxxx 

XXX 

X 

XXX 

13 

2 


X 


X 


2 

Totals | 

3 

8 

3 

3 

3 

20 


FIGURE 4. 


Check Sheets 












Cause and Effect Diagram 


The C&E (Cause and Effect) Diagram is the product of brainstorming. As shown in 
Figure 5, the problem is identified at the right end of the main spine. The QC Circle 

through brainstorming identifies possible causes of the problem. Generally, these 

causes can be classed under subbones of method, men, machine, and material. Although 
not commonly considered to be a statistical technique, the C&E Diagram is an essential 
component of the problem solving process used by the QC Circle, because it is a method 
by which brainstorm data can be recorded and organized. 



FIGURE 5. Cause and Effect Diagram 


Graphs 

The Circle needs to be familiar with the purposes of several kinds of graphs and 
capable of preparing these graphs. These include: time series graphs of productivity, 
costs, percent defects; histograms; Pareto charts; pie charts; and bar graphs. Basic- 
ally there are two purposes served by graphs. First, the Circle uses the graph for 
analysis, to see what has happened. Secondly, the graph is one of the most effective 
devices for communicating information. Reporting writing by QC Circles is a very real 
and necessary activity, especially to keep management apprised of its accomplishment. 
(At one of the plants of Honda Motor Company the author was shown a section of note- 
books filled with QC Circle reports. The notebooks, side by side, filled about two 
feet of bookshelf and represented only a six month time period at that one plant!) 

Other Statistical Tools 


The QC Circle will employ various other statistical concepts such as statistical 
control charts, scatter diagrams, binomial probability paper, random sampling, and 

acceptance sampling. Dr. Ishikawa states that these techniques, as well as the 

previously mentioned ones, are to be used by QC Circles, "The techniques in this book 
are those which we feel should be known by all QC Circle leaders and, if possible, all 

Circle members as well."^^ A variety of other, more advanced statistical techniques 
are employed by some Circles, such as experimental design. However, these tools are 
less known and infrequently used by Circles. 

Degree of Training 

In Japan there is a massive educational effort to teach the methods and concepts 
of quality control and QC Circles. A brief description of some of the courses offered 

(18) 

by the JUSE will summarize this. 1 Two courses are designed for general foremen, 
foremen, and assistant foremen: a six day, QC Basic Course for Foremen, and an eight 
day advanced course, QC Application Course for Foremen. Both courses emphasize quality 
control methodology including the solving of problems using the statistical tools we 







have already mentioned. Both courses incorporate material on the QC Circle. The QC 
Basic Course for Group Leaders is a three-day course for assistant foremen, QC Circle 
leaders, and other leaders; it covers basic group problem solving. A variety of other 
courses are also available: a correspondence course; industrial engineering course; 
within-company courses; etc. 

In the United States a variety of courses have been available which teach many of 
the basic problem solving concepts. However, the clearly defined and complete problem 
solving methodology, which we described earlier, has generally not been available. The 
exceptions are noteworthy. First, Dr. Joseph Juran has covered the basic problem solv- 
( 19 ) 

ing sequence. Secondly, the QC Circle training done at Lockheed Missiles and Space 

Company and that done by Wayne Rieker^^ as well as Jeff Beardsley includes a one- 
to two-week training period for the QC Circle facilitator. The facilitator in turn 
trains the foremen or Circle leaders through a similar course. Finally, the Circle 
members, who are hourly paid employees, are instructed in a basic teaching/application 
course which may be given in one-hour sessions over a several month period. 


DEVELOPMENT OF NEW STATISTICAL METHODS FOR USE BY QC CIRCLES 

The development of new statistical methods for use by QC Circles has to meet 
several stringent conditions. First, the concept must be easily teachable to the Cir- 
cle leaders and members, hence it cannot be a concept which presupposes either a strong 
statistics or mathematics background. Secondly, it must be relevant to the kinds of 
tasks which Circles undertake (a non-parametric test for the study of stock market quo- 
tations would not be appropriate in a production facility). Thirdly, it must offer 
sufficient advantage over existing tools to warrant bringing it into training courses, 
journals, and other means of transmitting the concept to the Circles. Finally, the 
tool must be robust so that it will apply as a good approximation to situations where 
not all the conditions are completely satisfied. 

One example will be given of the kind of technique which may meet the above cri- 
teria. This involves a significance test on the Pareto analysis. The essential prob- 
lem is for a Circle to determine if there are sufficient differences between categories 
of the Pareto chart to justify the decision to choose the most frequently occurring 
item as the most important one. From a statistical viewpoint the concern is that the 
differences in frequencies be more than just random sampling variation. The problem is 
complicated statistically in that ordinarily the Pareto chart is based upon nominal 
data. Therefore many of the non-parametric tests based upon interval and ordinal data 
will not apply. A further complication is that the number of classes is indeterminate . 
Also, some Pareto charts are based on criteria other than frequency, such as dollars; 
this may prove to be a very knotty complication. 

Other potential examples of new theory and/or new applications of statistics by QC 
Circles include the following: simplified methods for analyzing designed experiments 
since the analysis of variance is too sophisticated for use by Circles; and the corner 
test for association which provides a simple significance test for correlation. 


APPLICATION OF STATISTICS TO QC CIRCLES 

In order to provide management with a viable, accurate model of the role which QC 
Circles play in industries and institutions, it is necessary that methods be developed 
to answer two questions. First, what are the effects upon QC Circles of various fac- 
tors: unionization, skill levels, size of firm, type of firm, type of work, culture, 
management philosophies, etc. Virtually nothing has even been attempted in evaluating 
how these factors affect QC Circles. To a large extent the problem can be handled only 
through surveys. Some factors, such as skill levels and type of work, may be amenable 
to designed experiments. 

A second type of factor which is expected to affect QC Circles is that over which 
management has some control. A good example would be training. At this point in time 
we do not have even a rough, quantifiable model showing the relationship of amount and 
type of training to success of the Circles. Because of the heavy cost of training, 
such a model would be of considerable value. This kind of factor should be relatively 
easy to evaluate using designed experiments. For example, within one particular indus- 
trial plant, new Circles could be randomly selected for differing training programs. 

( 22 ) 

A randomized block design seems appropriate here. 



The second major area of applications of statistics to QC Circles is to answer the 
question: what are the results of QC Circles? QC Circles can be expected to reduce 
rework, scrap and other costs. However, as with any program through time the effects 
of the Circles are likely to be confounded, or mixed in, with the effects of other fac- 
tors such as new cost reduction programs, business cycles, etc. It becomes incumbent 
upon the statistician to devise experiments and surveys which will effectively separate 
the two types of factors. This is by no means a trivial matter. There may be a need 
for factorial and fractional factorial experiments to adequately cover this. 

This paper has attempted to convey the extent to which statistics applies to QC 
Circles. The Circles themselves incorporate basic statistical tools as integral parts 
of their problem solving process. New theory and applications of existing theory need 
to be developed and placed into the kitbag of Circle tools. Finally, a great deal of 
intensive, sophisticated investigation is required, both to evaluate the degree to 
which various factors impinge upon the Circle and to quantify the effects which Circles 
have upon motivation, productivity and quality. 


REFERENCES 

1. Personal notes from the First Annual QC Circle Conference, Lockheed Missiles and 
Space Company, Sunnyvale, California, Nov. 2, 1976. 

2. Personal communications at Honda Motor Company, Japan, August, 1972. 

3. Ibid . 

A. Ishikawa, Kaoru, Ed., QC Circle Activities , JUSE (Union of Japanese Scientists & 
Engineers), Tokyo, 1968, (In English), p. 21. 

5. Ibid . 

6. Op. cit. , p. 3. 

7. Ishikawa, Kaoru, Statistical Application Research , Vol. 21, No. 4, Dec. 1974 
(published by The JUSE in English), p. 5. 

8. Personal communication from the JUSE, January 1978. 

9. Personal communication with Grupo Cydsa, Monterrey, Mexico, 1976. 

10. See QC Circles: Applications, Tools, and Theory , Ed. By D. M. Amsden and R. T. 
Amsden, ASQC, Milwaukee, Wisconsin, 1976, for a review of the Lockheed Missiles 
and Space Company experience. Also contact Sperry-Vickers , Jackson, Mississippi; 
Buick Division of GMC, Flint, Michigan; and Sanwa Bank, San Francisco, Calif. 

11. See "Integration of TQC and Motivation Programs," by Dr. B. Veen of AKZO Research 
Laboratories, Arnheim, Netherlands, published in the 31st Annual Technical Confer- 
ence Transactions , ASQC, Milwaukee, Wisconsin, 1977, pp. 266-272. 

12. Ishikawa, Kaoru, Guide to Quality Control , Asian Productivity Organization, Tokyo, 
204 pp. Order from: Books for Asia, Limited, 30 Tat Chee Avenue, Ground Floor, 
Garden City, Kowloon, Hong Kong. 

13. Fisher, Ronald A., The Design of Experiments , Hafner Publishing Co., Inc., New 
York, 1960, p. 3. 

14. Juran, Joseph, Managerial Breakthrough , McGraw-Hill Book Company, New York, Lon- 
don, Toronto, 1964, Chapter 4 "The Pareto Principle." 

15. Ishikawa, Kaoru, "Cause and Effect Diagram...," Proceedings of the International 
Conference oi. Quality Control, 1969 - Tokyo , JUSE, Tokyo, pp. 607-610. This is 
also published in: Amsden, D.M. and Amsden, R.T. , QC Circles: Applications , 

Tools and Theory , American Society for Quality Control, Milwaukee, Wisconsin, 

1976, pp. 91-94. 

16. Ishikawa, Kaoru, Guide to Quality Control , p. 31. 

17. Op. cit. , p. VIII. 

18. Ishikawa, Kaoru, Statistical Application Research . 

19. Juran, Joseph, Quality Control Handbook , 2 ed., McGraw-Hill Book Company, New 
York, 1962. 

20. Personal communication, Wayne S. Rieker, 14850 Baranga Lane, Saratoga, CA 95070. 

21. Personal communication, Jeff F. Beardsley, 4998 Harmony Way, San Jose, CA 95130. 


iwnu nwviv i b>wi inivnL wmi un&-nvb. i nrtiignv/i ivnu vi uvnw 


22. Davies, Owen L. , Ed., The Design and Analysis of Industrial Experiments , Oliver 
and Boyd, London and Edinburgh; Hafner Publishing Company, New York; 1960, 2nd. 
edition, pp. 145-198. 

23. This paper is heavily based upon "Applications of Statistical Theory and Method- 
ology in QC Circles," which was presented by R. T. Amsden at the 4th Annual 
Symposium on Uncertainty in Risk and Reliability, University of Bradford, England, 
September 2, 1978. 


LCS 020:10:000 


CONFIDENCE INTERVALS FOR POPULATION PERCENTILES 


Walter C. Thompson, Statistician 
Lorillard - Greensboro, N. C. 27401 


If one has a large size sample from a population that is normally 
distributed, the calculation of population percentiles is simple and 
straightforward. The calculation of the mean and standard deviation 
and the use of the table of standard normal distribution found in any 
statistical text will quickly yield the desired table of test scores 
vs. percentile of the population. This situation is, unfortunately, 
not the usual case. Data is expensive. Frequently the sample size is 
relatively small and the population distribution often departs from 
normality, particularly in the tails. The purpose of this paper is to 
present an algorithm for estimating population percentiles and confidence 
intervals for the percentiles that is independent of the population 
distribution and is suitable for use with relatively small size samples. 

COMPUTATION ALGORITHM 

The term test score is used here in the broadest sense. It might be 
the score attained by an individual on a written test such as the 
Scholastic Aptitude Test or it might be a test consisting of a tabulation 
of the number of hours absence from work by industrial employees. Other 
industrial applications include the distribution of machine parts usage 
for use in deciding stock inventory levels. 

1. Obtain the test scores for a random sample of size n from 
the population of interest. 

2. Rank the test scores from smallest to largest, i.e. 1 to n. 

3. Calculate the cumulative frequency corresponding to each 
test score, i . 

4. Estimate the population percentile corresponding to any 
given test score by calculating 100i/(n+l) . 

5. Calculate the confidence interval for the population 
percentile using the binomial confidence limits for the 
percentage calculated in Step 4. 

The estimates of population percentiles and confidence limits can be 
conveniently plotted on normal probability paper. The linearity of the 
plot of test score vs. percentile is a measure of the normality of the 
population if normal paper is used. A derivation of this algorithm is 
appended. 

DISCUSSION 

Our daily lives are greatly influenced by test scores that are 
reported as population percentiles. Reading comprehension tests and 
achievement tests mark our children's progress in school. SAT scores 
played an important part in our own educations and largely determine, if 
our older children will obtain college scholarships. Population 
percentiles are also used in industry. Some examples are productivity 
standards for employees paid on piece-rates, and hours of employee 
absence, a factor often used as a basis for performance ratings. 
Fluctuating parts inventory levels can also be viewed in this context. 

My experience indicates that tests like those named above are 
frequently published and accepted when they are based on a naive under- 
standing of statistical inference. The Central Limit Theorem allows us 
to use the normal distribution, with comfortable confidence, as a model 
© 1978 American Society for Quality Control 


for inferences concerning the mean. This generalization, however, does 
not extend to the parent distribution. It is prudent for us to exercise 
care in making inferences which are used by others in making decisions. 
The algorithm presented is nonparametric and therefore quite general. 

It is suitable for non-normal distributions as well as any that may be 
normal. The computation of confidence intervals for the quoted percen- 
tiles allows a ready assessment of the precision of inference made and 
prevents the undue assignment of importance to standards based on rela- 
tively small samples. On the other hand, the use of confidence limits 
also permits the use of small samples for limited inferences without 
being misleading. 

APPENDIX 

DERIVATION OF COMPUTATION ALGORITHM 

The cumulative density function, F , is the probability that some 
randomly drawn sample value, X, is equal to or smaller than stated 
value, x. 

F = Pr ( X * x) 

If we draw a random sample of size n from the population: 

Pr (All X 4 X ) = F n 

The probability that all X are equal to or less than x is the same as 
the probability that the largest sample value, X max , is equal to or less 
than x. 

pr <w * *> = f” 

Yl 

Therefore, the cumulative density function of X max is F . We wish to 
find the mean of this distribution. The probability density function 
is the derivative of the cumulative function: 


The mean, F, is the integral of the product of value and density: 

F = f F n F n_1 d F 
JO 

= n F n+1 / ( n + 1)1 

Jo 

- n / (n + 1) 

That is to say that, on the average, the maximum value of a sample of 
size n is located at the n/(n+l)-th fractile of the distribution. By 
a similar argument, it can be shown that the i-th member in an ordered 
sample is, on the average, at the i/(n+ l)-th fractile of the 
distribution . 

It should be noted that there was no restriction placed on the form 
of the distribution, F . Therefore, the rule is completely independent 
of the shape of the distribution. 
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HOW TO SELECT THE BEST CONTENDER 


Edward J. Dudewicz, Professor 
Department of Statistics, The Ohio State University 
Columbus, Ohio 43210 

Young Jack Lee, Ph.D. 

Biostatistics and Data Management 
National Cancer Institute 
Bethesda, Maryland 20014 

INTRODUCTION 

Managers and others in administrative positions are often faced with the problem 
of selecting the best contender , from a group of contenders . These contenders are 
often those remaining after preliminary testing or competition in a larger pool of 
candidates . 

The quality of a contender may be described in a qualitative way, or in a 
quantitative wsty. For example, if the contenders are types of lightbulb filaments, 
the best contender may be the type with the longest life; while if the contenders are 
regimens for treating a particular disease, the best contender may be the one with 
the highest response rate. Each of these cases involves contenders which can be 
characterized by a single quantitative parameter . Such cases have received much 
attention since the pioneering ranking- and- selection work of Bechhoferv 3 -) and 0 f 
Gupta vO in the early 1950*8. {For some references and discussion, see Gibbons, 

Olkin, and Sobelv3), KLeijnen(°J, or Dudewiczfe). ) Our interest here does not lie in 
such cases. 

The cases which do concern us here are those in which either multiple parameters 
characterize a contender, or qualitative ratings by ’'experts'* are sought and used as 
a basis for the decision. For example, oncologists base the assessment of therapeutic 
effectiveness of a cancer treatment method on such multiple parameters as: the 
curativeness of the method, the disease- free survival time of patients treated, the 
overall survival time of patients treated, the quality of life, the affordability of 
the treatment, and so on. As another example, consider the product selection problem: 
several versions of a new product have survived the developmental phase, and now one 
must be selected for marketing. Often a number of specially trained judges will then 
examine and rank the versions (from poor to excellent on a preference scale) on each of 
several characteristics (e.g., crispness, sweetness, skin texture, aroma, color appeal, 
and so on). In either of these cases (and many others) the question of how to combine 
the multiple parameters or multiple preference scores into a single parameter usually 
has no satisfactory answer. Our interest lies in such cases. (While the selection 
problem has an obvious solution in the situation where the same contender rates as 
best on all measures, that situation is so unusual as not to need further mention 
here. ) 

As the above examples show, definition of what it means to be the best contestant 
is usually not at all straightforward in multiple parameter cases. Below we will 
give such a definition , and discuss procedures for selecting the best contestant using 
our definition. In the remainder of this paper we will proceed as follows: first, 
we will discuss these problems in a practical example with real data; second, we will 
give our definition and procedure; third, we will give mathematical details of our 
procedure* s characteristics. 


EXAMPLE: THE WILCOXON AND YOUDEN PRIZES 
Each year, the Chemical Division of the American Society for Quality Control 
awards the Frank Wilcoxon Prize and the Jack Youden Prize. These Prizes are awarded 
for articles in Technometrics, the Wilcoxon being for the best practical application 
paper and the Youden for the best expository paper. In the 1976 competition (held in 
1977) for these prizes, all the articles appearing in the 1976 volume of Technometrics 
formed a pool of candidates and fifty referees selected, from the pool, six articles 
for the Wilcoxon award competition afd seven articles for the Youden award competition. 
(Both authors of this paper served the Awards Committee, one (EJD) being the Chairman 
of the committee. ) 

© 1978 American Society for Quality Control 
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Merit of each article depends on such qualities as originality, scientific con- 
tribution, readability, etc., and preference scores (e.g. from poor to excellent) on 
such qualities by referees are measurements to determine relative ranks of articles. 

The difficulty associated with the preference- score method is how to combine pre- 
ference scores into an acceptable parameter ranking articles. Thus the Awards Com- 
mittee instead asked referees to rank the final contenders from best to worst according 
to their judgement on the overall quality of each contender (referees were not 
mandated to complete the ranking), and as a result we obtained the imcomplete rank 
score matrices of Tables I and II. 


TABLE I 


TABLE II 


Rank Scores of 7 Contenders for the Rank Scores of 6 Finalists for the 


1976 Youden Prize* 


1976 Wilcoxon Prize* 


Referee 


T 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 


Cl c 2 c 5 C 4 c 5 C 6 c 7 

'4 2 1 5 "6 0 T 

2 0 5 1 4 3 6 

15 2 3 6 0 4 

2 5 6 0 3 1 4 

1 2 5 3 ^06 

6 2 3 4 5 0 1 

5 4 1 3 2 0 6 

2 1 0 6 5 3 4 

4 5 2 1 3 6 0 

6 2 3 1 4 0 5 

4 5 3 2 6 1 0 

2 3^0156 
2 6 .- 4 3 5 

2 - 3 6 4 -5 

5 4 - 6 2 -3 

4 5 - - - 3 6 

6 4 - 5 3 -- 

4 5 3 - - - 6 

4 5 - - 3 - 6 

365-4-- 

- 6 5 - - 4 - 

5 - - - 4 - 6 

- 5 - - 6 - - 

6 - 


Referee C 1 C 2 

1 3 4T~ 

2 2 4 5 3 1 

3 5 2 4 3 6 

4 5 6 2 4 3 

5 2 3 ^16 

6 3 4 2 5 1 

7 5 1 4 3 2 

8 2 6 3 5 4 

9 5 4 3 2 1 

10 6 5 1 2 4 

11 46-35 

12 6 4 5 -- 

13 - - 4 6 3 

14 3 5 - 6 4 

15 3 4 - 6 5 

16 436 - 

17 5 - 4 6 - 

18 - 4 6 5 - 

19 - 4 - 5 - 


5 

6 
1 
1 

5 

6 
6 
1 
6 
3 

3 

5 


5 


6 


*The referee numbers are arranged for 
convenience in preparing the tables. 


In principle one contestant is to be selected for each award (but if two or more 
contestants tie for a prise, then the prize is split instead selecting one from the 
tied contestants based on other considerations). 


The question now is how to utilize the incomplete rank score matrices to deter- 
mine the awardee (s) for each prize. There are a number of ways summarizing the in- 
complete rank score matrices and awardee (s) m^y be selected based on any of several 
summary statistics. We present several summary statistics for the Youden prize rank 
score matrix in Tables III, IV, V, VI, VII. 


TABLE III 

Sum of all Scores in Table I with 
Missing Scores Replaced by Average Scores 


^1 

TJo" 


~2 

"863" 


3 

TT 


4 

W 


5 

w 


6 

w 


7 

W 



TABLE IV 


Sum of Modified Scores (5 points for a first place vote, 4 points for a 
second place vote, ... ,0 points for a sixth or 
seventh place vote and missing scores replaced 
Ty average scores for referees 16 through 24) 

C, c s> c „ C . C c c C 
12 3 4 5 6 7 

56.62 64.42 40.62 3«*T 61. OB 28.26 69/2B 


TABLE V 

Sum of Modified Scores (3 points for a first place vote, 2 points for a 
second place vote, 1 point for a third place vote* "™™ ~ 

0 point for others and missing scores replaced 
' by average score for referees 23 and 24) 


1 2 3 ^ 5 6 7 

21.7 26.7 1'2.7 14,'/ 23. 5 9.2 33'* 7 


TABLE VI 

Sum of Modified Scores (2 points fg>r a first place vote, 1 point for a 


missing scores 

replaced by average score for 

C, C 

C, 


1 

2 3 

4567 


9.17 14.1™ "6. 17 ' 8.17 10.17 « 19^7 


TABLE VII 

Frequency of the First Place Vote 


'2 


From Tables III, IV, V, VI and VII we see that contender is obviously to be 

selected as winner of the Youden prize, even though Tables III and IV show that con- 
testant C 2 is very close. We next look at the Wilcoxon prize rank score matrix of 

Table II, and again obtain several summary statistics (in Tables VTII, IX, X, XI, XII). 

TABLE VIII 







TABLE IX 


Sum of Modified Scores (4 points for a first place vote, 3 points for a 
second place vote,...,0 points for othera and 
missing scores replaced by average scores for 
referees 17, 18 and 19) "* 


Sum of Modified Scores (3 points for a first place vote, 2 points for a 
~ second place vote, 1 point for* a third place vote 
and 0 points for all others) 


2 3 4 5 

21 12 22 13 


Sum of Modified Scores (2 points for a first place vote, 3 
second place vote and 0 points for all others) 


TABLE XII 


Frequency of the First Place Vote 


In attempting to determine the awardee of the Wilcoxon prize, we now have a 
problem: the summary measures in Tables VIII - XII do not all lead to the same selec- 
tion. While Tables VIII and IX indicate that should be selected and Table X shows 

that all the contenders except C and C are very close with Q having a slight edge 

3 5 o 

over others. Tables XI and XII clearly suggest that the Wilcoxon prize should be 
split between and Cg. In order to resolve this problem, we must now face two 

questions: "What do we mean by "the best contender"?," and "Given the definition of 
best contender, which of Tables VIII - XII should a selection procedure he based upon?" 

DEFINITIONS AND PROCEDURES 

Suppose that the number of contenders is k (k » 2 or 3 or 4 or . . . ) and that each 
expert ranks at least t (l < t < k) contenders from best to worst. Denote the i*h 
contender by 

Let denote the probability that contender is rated as the (k-jt+l)ll best, 

among C C , by any expert. Define 
X K 

k 

U,(t) * 2 (< + t - k) t u - 

1 i-k-t+1 


Call the contender associated with max(u^ (t), ••• > (t)) the best contender (by 

ranking) t-out-of-k , and denote that contender as C-best(t). (it is, of course, 
possible that different contenders are C-best(t) for different values of t, a fact 
related to the so-called "voting paradox". Hence we should not be surprised if 
different values of t lead to different selections by our procedures below. ) 

The selection procedure R(t) for finding the C-best(t) contender is: Assign 
scores t, t-1, . . . , 2, 1 and 'k - "t O’ s to the k contenders, using the rank score 
matrix of the n judges. (The higher the score, the better a contender is rated.) 
Denote by R^^t) the score assigned to by the referee. Let Vj_(t)=2 ®^Rj^(t), 

and select the contender with the largest of V. (t), . . . , V (t), 

X K 

®ax(V^(t), V 2 (t), ... , V k (t)), 

as the best. If two or more tie either split the first place or select one based on 
other considerations (or break the tie by randomization). The first two options are 
recommended for practice, (The last option is required for theoretical development 
of selection procedures below. ) 


The Youden award rank score matrix leads to selection of as the best con- 
tender, regardless of the choice of t. As for the Wilcoxon award, 
nuucfv^?), V 2 (5),...,V 6 (5))= V 4 (5), 

mwfVj/lt), V a (U),...,Vg(4))- V 4 (U), 

miucfVjte), V 2 (3),...,V 6 (3))« V 6 (3), 

■h(Ti(2>, V 2 (2),...,V 6 (2))=- Tg(a) - V 4 (2), 

max(V^(l), V 2 (l),...,Vg(l))= Vg(l) - V 4 (l). 

Thus either or or both are selected dependent on the value of t used (and/or 

whether or not randomization is done). Since (as will be shown below) t = 1 is in a 
certain sense superior to the other possible choices of t, it is reasonable to select 
both and Cg, and this was the actual decision reached. 

MATHEMATICAL DETAILS; RELATIVE EFFICIENCY; CHOICE OF t 
In the following we will assume that the best contender is the same regardless 
of the choice of t. (This assumption is needed because the choice of t, i.e. com- 
paring R(l), R(2), . . . ,R(k), makes sense only when the best contender is not defini- 
tion-specific (because if it is definition specific, so also must the procedure be 
definition specific), ) 

Letu[i](t) denote ordered unknown values of p^(t), 

i^g( t )»« * •»h lc (t) and the probability associated with The contestant 

associated with is unknown for each i and denoted by C^^(t). The strength 


of a selection procedure is often measured by the probability of correct selection. 
If V^j(t) denotes the statistic associated with C^(t), then correct selection 

(CS) means the event 

V( k) (t) - max (V 1 (t),...,V k (t)). 

If P(CS|R(t)) < P(CS|R(t’)) when we have n referees and k contenders, then proce- 
dure R(t* ) should be preferred in this situation. 


i»fo Houiy I cv^ni'iiv^ML. i nwiNoH^ i iuino — ^ ni^Huu 




For notational convenience only, and without Ices of generality, we assume that 
the k th contestant is the best. Thus the statistic and parameters associated with 
C k are simply V (t) and * . 

* K KJt 

We will now suppose that 

** 2 + k i “ d Vx 2 Vk’ 1 5 i, i' < k-1, (l) 

that is we assume that the probability that the best contestant is rated best by a 
referee is greater than the probability that he is rated as (k-X+l) st best (l<£S£-l) 
and is greater than the probability that another contestant is rated best. 

Let n, k, t, and X*, 1 < X* <», be fixed and suppose (l) satisfies 

+kk * “ d W 5 **W 1 ^ < k-l- (2) 

Then for t > 2 

inf P[CS|f ij ,R(t)]<p[cs!R(t),t kk =X*^=X*<f ifk ,^ ^-*^,1 < l,V < k-l 3 (3) 

and for t « 1 

inf p[cs|+ ,R(l)3- pfcslR(l), f * X*f * 1 <t <k-l3. (4) 

1 J kk Xk 

In fact Pfcsli. .,R(1) ] is a function of only , ... , ^ , and , and 

kk {k-ljk 2k Ik 

a proof of (4) is given by Kesten and Morse (5). (Note that without the randomisation 
the equality (4) is true only asymptotically, since as n •», the probability that 
randomization is required goes to zero. For a more detailed discussion, see Lee(7) 4 ) 


For large n an approximation to the right hand side of (3) and (4) is given by 

f ^(-a- ♦ n^k-iKak-t-DWx*)) d < 5 ) 


2(k+X*-l)(k-l)r(X*)a(X*) 


t» -l/2 p 

$60 = J (2ir) exp(-x /2)dx, 


•a*) - 

v(x*) 

MX*) = {1 + a S (X*)) 1/2 , 

r a (X*) - Var{n' l/2 (V t (t) - V^t))}, 


V(X*) - Cov{n'' L/2 (V^(t ) - vjt)), n 


(V . (t) - V (t)). 
k 2 


Equation (5) can be computed numerically using Gaussian quadrature. Approximation by 
(5) is fairly reliable, and was off by less than 0.01 for the cases studied. 

We can compare the selection procedures R(t) by computing (5) for each t with 
fixed n, k, and X*« For example, see Table XIII. 

TABLE XIII 

P[CS 1 Comparisons 


n 

k 

X* 

P[CS|R(k)l 

p[cs!r(i)i 

30 

5 

2.0 

0.731 

0.787 

30 

6 

2.0 

0.559 

0.728 

30 

7 

2.0 

0.470 

0.673 
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Instead of computing P[CS|R(t)] to compare R(t)'s for given n, k, and X*, however, we 
equate (5) to a given P* (l/k < P* < 1) and solve the smallest n needed to satisfy the 
equation. Denote that n by \ #t (X*, P*). The ratio n fe t (l*, P*)/^ t , (X*, P*), 

1 < t t* < k, is called the relative efficiency of R(t* ) with respect to R(t), de- 
noted by Eff[R(t’), S(t )]• If EfffR(t' ), R(t)J > 3, then procedure R(t' ) is at least 
as efficient as R(t). Of particular interest is“Eff[R(l), R(t)]. Since 
Eff[R(l), R(t)] requires a computation for each combination of (k, X*, P*), we in- 
stead compute and obtain 

lim Eff[R(l), R(t ) ]«&&[ . (6) 

X*->1 5 t( 2 k-t-l ) 2 

If t ® k, lim Eff[R(l), R(t)] * (k+l)/3. In our Wilcoxon prize example with k = 6 , 
X*+l 

( 6 ) is given in Table XIV. We thus see 


TABLE XIV 


t 

2 3 4 5 6 

lim EfflR(l), RTtU 

X*-*! 

1.30 1.67 2.04 2.33 2.33 


that R(l) is the most efficient procedure. 

Note that whether inequality ( 3 ) is strict or not, the conclusion obtained does 
not change. If inequality (3) is strict, then the efficiency given by ( 6 ) is a lower 
bound for lim Eff[R(l), R(t)]. We conjecture, however, that inequality (3) is not 
X*-*l 

strict. 
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Distributional Analysis of Non-Normal Multivariate Data 


Gregory F. Gruska 
Environmental Activities Staff 
General Motors Corporation 


Because of the present economic and sociologic environment, there is an increasing 
awareness and subsequent stress in industry of providing management with the most 
accurate and timely view of the state of the quality system. As an aid to proper 
decisions on the acceptability of the quality of design as well as quality of con- 
formance, it is frequently desirable to know the underlying distribution of pertinent 
quality characteristics. Although there has been much work recently to improve the 
application of (univariate) non-normal forms to industrial problems, these techniques 
do not consider the multivariate structure prevalent in many situations. 

This paper will discuss the availability of multivariate non-normal distributional 
techniques and examine their merits within an industrial environment. 

INTRODUCTION 


Beginning in the mid-sixties, there has been a rapidly increasing trend in consumer 
awareness and subsequently in demands for "bigger and better" products. This growing 
consumerism has, in turn, motivated politicians to introduce and pass legislation and 
regulations which impact all areas of manufacturing and service. Since some of these 
political rules may run counter to popular (individual) opinion, the manufacturer may 
be caught between conflicting goals. Yet he must do his best to satisfy both, often 
treading a fine line between politically and commercially acceptable. 

A basic but global example of this phenomena can be seen in the changes wrought by the 
various safety regulations. These regulations, aimed to protect the lowest common 
denominator (often from himself), can require extensive processing changes and/or 
additions to the product. This, in turn, increases the cost which may be more undesir- 
able to the consumer. 

Thus, the manufacturer must gather in a timely, efficient and effective manner informa- 
tion on the status of the product/process. This endeavor is complicated by the fact 
that a product/process rarely has only one characteristic of interest. Even if these 
multiple characteristics have all the "nice" properties (normality, mutually indepen- 
dent), application of tradiational quality control and univariate data analysis tech- 
niques tend to be conservative and therefore cost ineffective when applied to these 
multiple variable situations. 

The government is also finding itself in increasing similar predicaments. With every 
new governmental regulation and standard, new regulatory agencies must be formed or 
existing agencies expanded in order to ascertain the compliance of industry to the 
standard and subsequently to enforce the regulation when necessary. Because of budget- 
ary and political pressures, the agencies want to gather the requisite information in a 
timely, efficient and effective manner. 

Although the agency often can (and does) minimize the in-house cost of data collection 
by requiring the manufacturer to periodically gather and report information on a speci- 
fied sample, it is realized that this cost is ultimately born by the consumer via 
taxpayer. (Note that we have come fully round the circle back to the consumer.) 

The scenario is further complicated by the overlapping responsibilities of several 
semi-autonomous agencies. For example, on December 31, 1974, the Environmental Protec- 
tion Agency (EPA) issued proposed regulations concerning the selective enforcement 
audit of end-of-line automative emissions. This proposal specified an attribute sam- 
pling plan with a 10% AQL to be applied to each of the three major emissions constitu- 
ents: hydrocarbons (HC) , carbon monoxide (CO), and oxides of nitrogen (NOx) . If this 
regulation was implemented as proposed, an audit would be successfully passed if each 
individual characteristic separately passes. If these vehicles were scheduled to be 
sold in California, they would also have been subjected to the California Air Resources 
© 1978 American Society for Quality Control 



Board's end-of-line emissions audit which required at least 10% of the vehicles tested 
to be under the standards on all three major constitutents . 

Both of these audits have the objective of assuring that vehicles being produced satis- 
fy, at the end of the assembly line, the standards to which they were certified. Yet a 
manufacturer could have successfully passed one audit without gaining any assurance 
that he could pass the other audit.* [GRU 2] Additionally, these audit plans did not 
utilize the variables nature of emissions’ measurements nor acknowledge the fact that 
these measurements are nontri.vially correlated. 

An "obvious" approach for any of the above situations is to utilize a variables sam- 
pling plan based on an appropriate non-normal multivariate continuous distribution. 

The problem, then, becomes to determine which sampling methodology should be utilized 
and how should this methodology be implemented within the industrial environment. 

MULTIVARIATE ANALYSIS 

In a broad yet actual sense, multivariate analysis encompasses practically the whole of 
statistical theory since any single sample can be considered as a special case of the 
multiple variate case in which the individual variables are independent. Also, in 
"traditionally" univariate problems, such as the test of equality of two means by the 
student T statistic, the primary idea (of multivariate analysis) of independence and 
the complementary idea of dependence is required. 

Yet most people view multivariate analysis in a narrower sense as that "branch of 
statistics dealing with the summarization, representation and intepretation of data 
sampled from populations where the variable elements yield measures of more than one 
characteristic" [KRA 1] . Examination of the whole field of multivariate analysis 
yields two primary features: 

o Consideration of a set X = {x^, -----, x } where each x. is an element of the 
same p space (i.e., consists of p variables): 

o The variates are dependent among themselves and must be considered together; an 
individual (or group of) variate cannot be isolated and considered independently. 

The need to identify the dependence and/or interdependence relationships in a set of 
variates has resulted in five general categories of multivariate analysis. 

o Factor Analysis 

o Component Analysis 

o Discriminant and Classification Analysis 

o General Linear Multivariate Models 

o Distribution Analysis 

The history of multivariate analysis is usually taken to have begun with the works of 
Francis Galton. Although the bivariate normal distribution has been studied throughout 
the nineteenth century, interest in multivariate analysis remained relatively dormant 
until it was stimulated by Galton. As early as 1889 he was applying the mathematics of 
the bivariate normal to studies of generic inheritance. He did not introduce any new 
forms of joint distributions himself, but he developed the idea of correlation and 
regression as expressions of linear relatedness of two variables and focused attention 
on the need for greater knowledge of possible forms of multivariate distributions. 
Besides being a pioneer of multivariate statistics, Galton is considered to be the 
founder of trait and factor psychology for which this methodology is required. 

Subsequent development was channeled primarily by this type of use as well as by compu- 
tational capabilities. The main emphasis was on non-engineering type problems (e.g., 
in biometrics, education, agriculture, sociology, psychology, etc). An underlying 
multinormal distribution was assumed by most investigations. This should not be sur- 
prising do to the complexity of multivariate forms and the fact that most calculations 
at this time were performed manually. 

As an indication of the impact of this focusing of research to essentially the first 
four categories above, less than 7.5% of the entries in the extensive bibliography of 


*Although the California Standards are presently lower than the corresponding Federal 
Standard, the statutory limits are the same. Thus the audits can be compared, assuming 
the same standards. 



multivariate statistical analysis by Anderson, Gupta, and Styan [AND 2] deal with non- 
normal multivariate forms and analysis. Most of the entries dealing with these matters 
are highly specific and limited in application. In fact, the number of engineering 
applications, whether multinormal or non-normal are very limited. For example, less 
than a dozen Quality Control oriented papers can be found in the multivariate litera- 
ture . 


This seeming lack of interest in distributional analysis is unfortunate since although 
the engineering sciences can utilize elements of the first four categories above, it 
requires distributional analysis techniques. 


Distributional Analysis 

Any examination of available multivariate distribution analysis will show that the 
dominance of the multinormal distribution among multivariate distributions is more 
marked than that of the normal among univariate distributions. Because of this prepon- 
derance of literature on the multinormal, approaches in describing its distributional 
form are many and diverse. One of the more straightforward descriptions is that a set 

of random variables X^ , , X m have a joint multinormal distribution if their joint 

probability density function can be written in the form 


p x 04, 


. x )= 


C * exp [-(positive definite quadratic form in X,, X )] (1) 

i m 

with C an appropriate constant. If the exponent is written as - |^(X - 4)' A(X - 4) 


where A is a real symmetric positive definite matrix, then C must be a function of 4 
and A and, in fact 


C = (2n)“ (1/2:)m | V| 

-1 


• 1/2 


where V = A * is the variance-covariance 
be written as 


(dispersion) matrix of X. 


Therefore (1) can 


P x (X , — X m ) = (2n)" (1/2)tn |V|‘ 1/2 exp {-(1/2)(X-|) V^X-I)} (2) 

which is the "standard" raultinormal form where V has the meaning given above and 4 = 
E(X). 


If A is only positive semi-definite (i.e. IAI = 0), the joint distribution of X., , 

X is called multinormal. 1 

m 

Because of its form (1), the multinormal has the property that if any subset of vari- 
ables is eliminated (by integration) the remaining variables have a joint density 
function of the same form. 


In addition to these properties, the multinormal shares with the univariate normal the 
characteristic that the (joint) distribution of any linear function of X’s will be a 
(singular or nonsingular) multinormal distribution. 

Full accounts of the history and development of the multinormal can be found to some 
extent in virtually all books on multivariate statistics. Most notable are those 
written by Anderson [AND 1], Dempster [DEM 1], Kendall [KEN 1] and Johnson and Kotz 
[JOH 1]. 


As in the univariate case, research in the multinormal distribution has generated "spin 
o|f" research into multivariate sampling distributions analogous to the univariate t, 

X (or gamma) and F (or beta). That is given a random sample of size n from an m- 
variate multinormal population, the observations can be represented by n independent 

vectors x. = (x. ., , x. ) with probability density function given in (2) . The 

maximum likelihood estimators of the elements of V are the corresponding elements of 

n where 


s . . 
ij 


n 

= I (x. 


3J 


X. . ) (x.. - X., ) 

j ik k 



and 


x. . = 
j 


n 

I 

i=l 


For the univariate case (m=l) the distribution of S = is just x with v = n-1 
degrees of freedom. 2 Thus the joint distribution of S is regarded as the generalization 
of the univariate x (or gamma) distribution. It should be noted that this is not a 
multivariate gamma distribution since these have all marginal distributions of the 
gamma type while for j ^ k, s.^ does not necessarily have a distribution of the gamma 
type. J 


The joint distribution of the variance-covariances for m normal variates was developed 
by Wishart [WIS 1] in 1928 and is called a Wishart distribution. Fisher had previously 
(1917) obtained similar results from the bivariate case. Although latent in Wishart's 
work, the characteristic that the dispersion matrix was the natural extension of the 
variance in univariate theory to the multivariate case was brough out by Wilks in 1932. 
Consequently one line of development has been the study of ratios of matrices of the 
dispersion type, this being the multivariate analogue of the t distribution, F distri- 
bution, etc. 


As before, suppose X = (x^, , x^) is multinormal with mean vector 4 and dispersion 

matrix V. Let S, S* be independent Wishart matrices also with the dispersion matrix V, 
based on n, n* degrees of freedom, respectively. Then if T is a mxm matrix such that 
TT ' = S, natural candidates for the multivariate analogue of t and F are 


t = T' 1 (x - 4) (3) 

F = T" 1 S* T'" 1 (4) 


Although (3) can be shown to be equivalent to the general multivariate T distribution 
it is by no means the only candidate. Similarly with (4) and the multivariate beta. 

Additional generalized multivariate beta distributions have been described by Waal 
[WAA 1] , Mitra [MIT 1] , Khatri [KHA 1] , Krishnaiah [KRI 2] , and Bennet [BEN 1] to 
reference a few. 


This same type of multiplicity of forms is also evident for the multivariate gamma 
distributions. For example, see Sarmanov [SAR 1], Eagleson [EAG 1], Moran [MOR 1, 2], 
Bose [BOS 1] or Krishnaiah et al [KRI 3]. 


Considering all these different forms each stating to be the generalized multivate 
beta/gamma, it is ntural to question if there is a general ’’generalized” multivariate 
beta/gamma. Since the univariate Pearson family of distributions encompass both the 
univariate beta and gamma distributions (cf Gruska, Mirkhani and Lamberson [GRU 1]), 
a multivariate equivalent of this family would be a reasonable starting place. 

Extension of the Pearson family to the bivariate case has been exhaustively described 
by Van Uven [UVE 1, 2] and reported by Johnson and Kotz [JOH 1]. The generalization is 
accomplished by replacing the differential equation for the univariate family 


1 dy 
y dx 


a + x 

2 

c +c..x+c„x 
o 1 2 


(16) 


by the pair of partial differential equations 

(j = 1, 2) 


a i° s y = 

3x . 

J 


vw 


(17) 


where L. and Q. are linear and quadratic functions respectively. By equating 

the two^ different expressions for 8 log y/Sx^Sx^, it can be seen that the L/s an< ^ Q^’s 

cannot be chosen in a completely arbitrary manner but must satisfy the relationship 
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The marginal distributions of these joint density functions can be represented by 
Pearson type curves and it is possible to fit the parameters of these curves by the 
method of moments . 

In fact, none of the multivariate density functions --including those not described here 
such as the noncentral versions of the multivariate , F, beta, or gamma, series expan- 
sions etc. — are feasible choices for a general (wide applicability) non-normal multi- 
variate distribution. The reasons for this are primarily 

a) functional complexity requiring extensive numerical calculations not only for 
parameter determination but also for evaluating the corresponding distribu- 
tion function. 

b) unlike the multinormal, it is not possible to replace a pair of correlated 
variables by a pair of independent ones (using a transformation corresponding 
to a rotation of axes) in order to simplify the analysis. 

The first problem (a) may be minimized by considering multivariate distribution func- 
tions while (b) may be handled by multivariate translation systems. 

One univariate distribution function which is used extensively in the areas of relia- 
bility and stress/fatigue analysis is the exponential distribution 

F(x) = 1 - exp (-x/0 ). (x > 0; 0 > 0) (4) 

o o 

Bivariate analogues of this form has been developed by Nagao and Kadaya [NAG 1] , 

Gumbel [GUM 1], Moran [MOR 2] and Freund [FRE 2]. Weinman [WEI 1] extended Freund's 
bivariate exponential distribution to a multivariate exponential distribution by con- 
sidering the reliability of a system having m identical components with time to failure 
Xp , X^, each an exponential distribution of form (4). 

1/C 

Since the distribution of X' is Weibull if X is an exponential, multivariate 
Weibull distributions can be construced as the joint distributions of 

X, , , X 7 m where X,, , X have a joint multivariate exponential 

distribution. 


The multivariate Weibull distribution can also be approached through the extreme value 
distributions. The extreme value distributions are obtained as limiting distributions 
of the greatest (or least) values in random samples of increasing size. Extension of 
this concept to the bivariate case (i.e. considering the joint cumulative distribution 

function of X and Y ) have been obtained by Geffroy [GEF 2], Gumbel [GUM 4), 
max max 3 3 ’ 

Sibuya [SIB 1], and Oliveira [OLI 1]. Unfortunately, a general multivariate extreme 
value distribution has not been developed. 

Another useful univariate distribution function is the Burr distribution (see Burr 
[BUR 1], Gruska, et al [GRU 1]) which has the form 

F(x) = 1 - (1 + x C )' k (c,k > 0; x > 0). (5) 


The joint (unconditional) density function of a multivariate Burr is (Takahasi [TAK 1]) 


p(x) 


T (k+m) 
T(k) 


r ~(k+m) 
i + i|d i c j | 
i 1 ■> 


m 

n 
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Of .c ,x. c j-l 
J ] J 


( 6 ) 


Takahasi also showed that any subset of the X's have a joint density function of this 
same form (6), with appropriate changes in the parameters! 


One of the disappointments in collecting the above information is that the literature 
search has not uncovered any quality control methodology based on non-normal multi- 
variate distributions. Any quality control methods reported relied on the charac- 
teristics of the multinormal distribution. 



One major advantage the multinormal distribution has over its non-normal bretheren is 
model simplification ability (b above). This has assisted researchers in developing 

niSLs S6 ft wou?d S be aSe t °V ^ multinormal * In order to utilize this wealth of Lch- 
Tntn l' nat * ral to att ^pt to transform a general multivariate population 
n o a multinormal population. Consequently, research on this subject can follow one 


of three distinct paths: (1) development of requisite techniques for a specific multi- 
variate density function(s); (2) development of techniques for a specific multivariate 
distribution function(s); (3) development of methods to translate non-normal multi- 
variate data into multinormal data and development/extension of multinormal QC tech- 
niques. Of these, the third seems most tractable from a computational and implemen- 
tation standpoint. 
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EXECUTIVE TECHNIQUES FOR SNOW REMOVAL 


Nathaniel L. Geary, Director 
Industry and Government Relations Staff 
Office of Compliance 
Food and Drug Administration 
Public Health Service 

Department of Health, Education, and Welfare 


What has the Quality Manager in common with good reporters and 
good intelligence operatives? He has a need to know. He has specific 
requirements for information. Like a reporter or an intelligence 
operative, he needs a reliable source of information. The manager of 
quality, however, should have an advantage over the reporter and the 
intelligence operative simply because his company has developed a fine 
quality philosophy, a good set of quality objectives, practical oper- 
ational manuals repleat with reporting and feedback mechanisms, an 
excellent set of specifications for products and processees, and the 
most sophisticated testing and sampling protocols required to do the 
job, yet with all of these techniques, the manager of quality finds it 
necessary to evaluate the quality program periodically to assure himself 
and his management that the right things are being done, and that they 
are being done properly. How does the manager get his job done? How 
does he use the questions - Who? What? Where? When? Why? and How? 
to help him. Today, I am sharing the session with Bill Golomski, who 
will be discussing a similar subject in regard to consulting firms. I 
can assure you that we did not collaborate on selecting topics for this 
presentation, nor have we shared our speeches prior to the deadline for 
publication in the Annual Technical Conference transactions. Both 
papers relate to aspects of communication which need to be highly 
developed by the Quality Manager, or by the user of external consulting 
services . 

Let's review a few concepts before I launch into my socratic 
process. We have made assumptions that quality documents have been 
prepared and quality systems are in place within the company. A 
company may be a manufacturer of goods. It may be a purveyor of 
services, or, as in the case of Government, it may be a non-profit 
publicly owned corporation which is in the business of regulation or 
purveying services. The discussion today applies to all of these 
enterprises. In each enterprise, there is a balance of resources which 
is applied against a task or program. In each, the dollar is a common 
factor. In each of the enterprises as well, there is a determination of 
success or failure based upon the delivery of goods or services within 
a specified time period. In each, too, there is an assessment of the 
quality of output judged against specifications. It is necessary to 
over-simplify in order to cover the subject within the time allotted. 
Therefore, I have restricted my comments today to a discussion of 
Executive Techniques for Snow Removal" in the management of quality. 

You will agree that there are aspects of quality management throughout 
all phases of the operation of an enterprise, yet all enterprises do not 
have staffs dedicated to the management of quality. What is being said 
here today in regard to the management of a quality operation may be 
applied within any enterprise which is interested in improving its 
operation . 

Let’s find out a little about ourselves. Let us ask ourselves - 
How have we designed our jobs? What qualifications must our employees 
have in order to do the jobs? Where can we find people with those 
qualifications? What do we have to do to qualify them? Who will train 
them? Who will supervise them? What is the quality of our supervisory 
talent? Where do we get our supervisors? After we have answered these 
questions, it is possible to look more specifically at the management of 
quality in terms of Who? What? Where? When? Why? and How? What do 
© 1978 American Society for Quality Control 
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you expect of your employees? How can you optimize their performance? 
Have you provided them with proper tools, proper training, proper 
supervision, and appropriate follow-up? Have you challenged them? 

Have you made them aware of the responsibility that they have for 
accomplishing the job? Have you encouraged them to think about what 
they are doing, and how what they are doing relates to the goals of the 
company? Do you encourage them to furnish honest, forthright reports, 
regardless of the consequences, or do you merely want them to give you 
the good news? Do you know their individual strengths and weaknesses? 

Do you check up on them to see that their reports are complete and 
accurate? Do you require certification or qualification of your 
personnel? Do you perform checks on their performance to assure that 
they know what they are to do, that they are doing the right thing, and 
that they are doing it properly? To whom have you delegated responsi- 
bility for supervision? Have you informed your employees and your 
supervisors what the checks and balances are? When procedures will be 
executed? And why? Have you advised them of the consequences to the 
company, to your department, and to them for failure to comply with the 
rules and regulations, specifications, SOP;s, etc.? Have you assured 
that the Labor Management Agreement reflects your requirements for 
personnel? Is the labor organization attuned to your requirements? 

What do they do to assure compliance? 

Quality Assurance functions, of course, vary among companies and 
among departments within a single company. Some quality functions are 
scattered among diverse departments: for example, Research and 
Development; Quality Control or Quality Assurance; Purchasing; Distri- 
bution, and perhaps others. What have you done to assure that the 
quality objectives of the company are translated into action within all 
other departments having quality functions? What have you done to 
assure that the quality requirements of the company are followed? Who 
in those other departments has quality responsibilities? To Whom do 
they report? How do you assure that the quality objectives are met? 
Certainly, the quality function must be managed, but it is not entirely 
clear to all managers and all companies that quality is all pervasive. 

It is necessary for the manager of a quality function to assert himself, 
to assure that his boss knows that the new plant being built down the 
road meets the requirements for a quality environment in which a 
quality product is to be made. He needs to know that the proper 
equipment is being used to produce that product. He needs input to the 
training programs, and even the hiring practices to assure that the 
proper people are selected for the job. All of these activities are 
preventive in nature. They are designed to minimize the impact of 
defective personnel, defective product, defective equipment, defective 
processes, procedures, etc. Certainly, prevention activities have 
costs associated with them. How do you compare your costs of preven- 
tion with the other costs of quality? How do you allot your funds in 
one area vs. another? What should your payoff be? Where will you 
realize the profit or the loss? It seems essential that one have at 
his disposal a system for determining quality costs. It seems equally 
important to have cost centers built into the accounting system which 
will capture these costs. What can the Quality Manager do to get such 
accounting systems installed? How can he justify the existence of such 
a system? What advantage would it have to company management? 

Such systems may improve the ability of the Quality Manager to 
determine accurately the costs of quality. Paralleling this system, and 
perhaps integrated into it, may be an audit system which helps to 
answer the questions - What is the organization doing vs. what should 
the organization be doing visa-vis quality management? What is the 
status of the quality control system? What are the results of quality 
control tests? What is the quality of product or service? What pro- 
visions are there for making alterations when defective systems are 
detected? How often are systematic reviews made of the entire oper- 
ation for effectiveness? What are the most likely areas for perfor- 
mance or cost improvement? The Quality Manager must answer these 
questions because in a sense, answers to these questions and questions 
like them, justify his existence. 
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The Quality Manager must learn to ask the hard questions - 
questions designed to elicit a meaningful response about a product, a 
process, a test, a procedure, a report, or any other aspect of the 
quality assurance operation. He should not ask the technician - Did 
you do the test? - because he would get a simple non-definitive response 
of either "Yes" or "No." It would be much better for him to ask the 
question - What test did you perform? What method did you use? How 
did you conduct the test? - to elicit a specific response to the 
question. The manager may further ask, incorrectly - Was the test 
within specifications? The answer may be a non-definitive "Yes" or 
"No." It would be much better for him to ask - What were the test 
results? How did the results deviate from the standard or the speci- 
fication value? Who performed the check analysis? I think you realize 
that snow removal techniques are simple means to get specific answers to 
questions which enable the Manager to learn or to obtain specific infor- 
mation about an operation. The Manager holds the key. If he asks a 
question that is not designed to elicit the response, he snows himself. 
Employees may also learn very quickly that if the Manager does not ask 
the right question, they do not have to be very precise in their 
response, and they may tend to give vague answers, non-specific 
answers to the questions. They learn very quickly how to snow their 
boss . 
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WHAT TO EXPECT FROM AN EXTERNAL CONSULTING FIRM 


William A, Golomski 
W. A. Golomski & Associates 
59 E. Van Buren Street 
Chicago, Illinois 60605 


There are efficient ways to work with consultants and excellent ways to get the 
most out of consultants when you have fuzzy objectives. 

The problem in many organizations is that it is considered a sign of weakness to 
admit that assistance is needed internally. To suggest that an external consultant is 
needed is even worse. 

Organizations use consultants as: 

1. Sources of information. 

2. Designers of systems and products. 

3. Temporary bodies for hire. 

4. Experts to solve problems or as sounding boards for ideas. 

Marketing research and patent searches are typical examples of using others for 
getting information. Simple survey work can be put out on a bid basis. Specialized 
depth interviewing or literature search is best done by those who specialize in it. 

Quality Control work is often in the system design area. It requires more than 
someone who has worked in the field for years. You also need the ability to establish 
rapport with others quickly. It requires creative ability to design a system that fits 
the organization, rather than apply one that worked elsewhere, but may not apply. 

An external consultant should: 

1. Require you to state the study objectives and time of 
completion clearly. 

2. Require you to "condition" the organization for your arrival 
unless "surprise" is the consulting strategy. 

3. Use talent in the organization to assist in planning the 
details of the study, if appropriate. 

4. Give periodic reports of findings during the course of the 
study. Perhaps some can be implemented without waiting for 
the study to be completed. 

5. Report on uncooperating people who were supposed to cooperate; 
but he should give more than a modest effort to obtain cooper- 
ation first. 

6. To report on problems or potential "problems" created by the 
consultant . 

7. Submit a report which addresses itself to the objectives of 
the study. 

8. To make sure that systems, practices and methods recommended 
are practical and implementable. 

9. To assist in implementing if that is part of the contract. 

10. To clearly state the fee schedule, expenses likely to be 
incorrect, an upper limit for the entire project, and how 

and when invoices will be submitted, and the terms of payment. 

11. To give references, but not copies of reports done for others. 

Don’t expect a consultant to have experience in your industry. It isn’t that 
essential. The important thing is to have worked successfully on problems such as you 
are facing. 

Expect a consultant to tell you who in the firm will be working on your project, 
what their backgrounds are, and the approximate number of days each person will spend 
on the job. 
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Don't expect a consultant to sign the standard R&D agreement, if he is designing 
a sampling plan. This is an unfair limitation of the consultant's ability to build a 
professional practice. 

Some consultants require a penalty for aborting a project. Try to get this pro- 
vision removed, unless you have persuaded the consultant to enlarge his staff for your 
benefit . 

Determine the number of copies of the report that you will get and state whether 
or not a special presentation on the results is to be given. If more are required, 
expect to pay for them. 

Fees today range from $ 150-$200/day for retired people who are trying to keep from 
being bored or to supplement their income to $350/day for younger professionals to 
$500/day for experienced consultants to $750/day principals to $1500/day for those with 
rare specialized knowledge. 

Training is usually at $1000-$2000/day for groups, if unique materials are to be 
prepared. A few lesser-equipped consultants will use the standard fee schedule given 
previously. 

Consultants are in a peculiar position. They are brought in to solve problems and 
to make you stronger for having been there. Beware of those who want long contracts 
and large retainers. This might be needed, but it might be self-serving. 

Consultants must be candid, and this may hurt. Make sure that they aren't destruc- 
tive. Consultants should be articulate in their written reports and in their work. 

They must be able to sell concepts and ideas. 
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A TOTAL QUALITY CONTROL SYSTEM FOR FOOD PRODUCTION 
IN HOSPITALS 

David E. Douglas, PhD, PE, CQE, CDP 
Associate Professor 
Department of Industrial Engineering 
University of Arkansas 
Fayetteville, Arkansas 72701 

Sharon H. Weatherly, MS, RD 
Coordinator, Dietetic Internship 
Veterans Administration Hospital 
Little Rock, Arkansas 72114 


INTRODUCTION 

Food and the cost of operating a hospital food service has become 
a major expense for hospitals today. Of the amount spent by a hospital 
on the food service, approximately one- fourth of this is spent on 
incoming raw materials or food. Although quality control presently 
exists in some hospitals in the form of standardized recipes, standards, 
spec i f i ca t. i ons , guides, etc., they lack a well-defined quality control 
program to insure that meals are served into which quality is designed, 
produced, and maintained at the most economical costs which allow for 
full patient satisfaction. The fulfillment of this total quality 
control concept requires the accomplishment of the four jobs of quality 
control: (1) new design control, (2) incoming material control, 

(3) food production control, and (4) special process studies. These 
jobs of quality control are achieved by (a) setting standards, 

(b) appraising conformance, (c) taking corrective action, and (d) plan- 
ning for improvement. 

The following paper presents a description of the jobs of quality 
control as proposed for implementation. Much more detailed work than 
can be shownin this paper is in the process of being developed, but 
food production control is the main emphasis for this paper. A pro- 
posed organization for quality control is also presented. 

NEW DESIGN CONTROL 

When one thinks of design, he usually envisions some part, sub- 
assembly, or complete system that will be blueprinted for production. 
Similarly, in a food service environment, the design for production is 
in the form of a written menu instead of a blueprint. At first glance, 
it appears that this is a problem for dietetics and does not involve 
anyone in the quality control department. However, several reasons 
necessitate that quality control be involved in developing new menus 
from initial menu development through production. 

The first and most important case for quality to be involved in 
menu writing is to set the standards both for the menus themselves and 
for the raw mater ials . Since the quality control department has the 
responsibility of appraising conformance to standards, it is important 
that the standards are such that appraisal is both possible and economi- 
cal. If a new menu requires the purchase of a raw material for which 
standards have not been established, then these must be determined. If 
a. vendor is involved, then acceptance procedures and vendor certifica- 
tion^ possibilities need to be explored. Production of the new menu may 
require new measuring instruments to determine conformance to standards. 
The quality control department would be best suited for determining 
which measuring instruments are needed, or providing alternate standards 
which eliminate the need for this equipment. 
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A second reason for including the quality control department in 
menu writing concerns statistics. The need for the generation of new 
menus can be the result of several factors, but one of the more common 
is patient desires. The quality control personnel should be familiar 
with statistical procedures and can help provide statistically sound 
forms and procedures for evaluating patient preferences concerning 
food and food service. 

Providing an organizational feedback loop for corrective action is 
the last argument for including quality control in the menu writing 
process. Since the quality control department is responsible for con- 
formance to standards, it provides an organizational structure for 
providing management with deviation problems with the new menu. Manage- 
ment can then take the necessary corrective action, and the experience 
gained should help eliminate the same types of problems for future 
menus . 

INCOMING MATERIAL CONTROL 

It is usually true that a food product's ultimate quality is deter- 
mined by the quality of the raw materials used to prepare the food pro- 
duct. Thus, quality control of incoming materials is very important. 
However, most hospitals receive such a large range of raw materials that 
if every single raw material, including those used in insignificant 
amounts, is to be subject to detailed examination and testing at 
frequent intervals, the cost will outweigh the advantages gained. It, 
therefore, follows that the dominant raw materials are selected for 
priority of attention and the selected materials are tested in relation 
to their contribution to product quality. 

The VA has already established standards for most incoming raw 
materials . (1) Quality control's part is to design forms for inspection 
and establish the economical frequency of inspection. A quality audit 
should be made randomly to ensure the accuracy of inspection and that 
inspection is at the proper intervals. 

Acceptance sampling procedures should be considered for some raw 
materials . There are two published sets of sampling plans for 
attribute sampling that should be considered. They are the MIL-STD-105D 
tables and the Dodge-Roming tables. One should become familiar with the 
philosophy toward protection before selecting a set of plans. Although 
not necessarily the best for a given application, the MIL-STD-105D is 
more widely recognized and used than the Dodge-Roming. The MlL-STD-414 
tables can be used when variables sampling plans are needed. Variables 
sampling plans should be considered since smaller samples are needed 
when compared to attribute plans. 

Although a large portion of the raw materials are purchased from 
government supply, some purchases involve vendors. Good vendor rela- 
tions are important to insure the desired quality. A quality record 
should be kept for each vendor used and vendor certification needs to 
be encouraged. Certification by vendor of the quality of his product 
results in both commercial advantage to him and a minimum inspection by 
the hospital. 

Most hospitals do not have adequate testing and measuring instru- 
ments. Most hospitals have scales and thermometers but, at a minimum, 
need a ref ractometer , brix hydrometer, and fat analyzer. Also, some 
hospitals do not have a testing laboratory for food, but do have labora- 
tory facilities for microbiological testing. 

FOOD PRODUCTION CONTROL 

There is an old saying among quality control managers, "Quality 
cannot be inspected into the product." This means that quality control 
must exist as the food is being prepared. It also implies that the 
individuals who prepare the food are the most important determinants of 
quality. Management needs to stress the importance of quality and 
develop programs to reinforce good quality both in employee training 
programs and inspection techniques. 
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Quality is objectively measurable even though many food attributes 
are subjective. Quality is measured in terms of a product’s chemical 
and physical attributes: flavor, texture, color, appearance, consis- 
tency, palatability , nutritional values, safety, ease of handling, con- 
venience, and portion size. The administration of the techniques of 
pre-meal evaluation for taste control and control charts for portion 
size are being developed by Banik and Mathews. 

Before pre-meal evaluation can be implemented, a well defined pro- 
gram for sensory evaluation needs to be developed. This includes: 

1. Developing a taste panel evaluation form 

2. Procedure for setting up samples to be tested 

3. Procedure for conducting taste panel and selection 
of panel or judges 

4. Procedure for training taste panelists 

Since taste is a much more subjective matter, being a combination of 
taste, smell, and feelings, training is a necessary first step to sen- 
sory evaluation. Judges learn ho w to handle food when tasting and to 
recognize and separate the various responses. They become acquainted 
with the terminology meaningful to the taste panel. 

The pre-meal evaluation is a prime technique to stimulate employees 
interest and concern in consistently meeting quality .standards over a 
long period of time. In this case, the pre-meal evaluation is being 
conducted in two different ways. First, all menu items at the noon and 
night meal need to be tasted to determine acceptability for patient con- 
sumption. Data is maintained on problem items. Five menu items are 
selected (on a random basis) , placed on a table and tasted by designated 
individuals. The taste panel evaluation form (Figure 1) is filled out 
for these products, kept on file, statistics calculated and a p chart 
constructed. In this manner, statistical quality control can be main- 
tained on food items, but does not take up a tremendous amount of time. 
Using this method, quality can be evaluated on new, problem, and reg- 
ularly used menu items. 

Portion control is not only related to quality (nutritional value) 
but also very closely connected \vith the administrative problems of pur- 
chasing, inventory, and production control. In anticipation of imple- 
menting a Computer- Ass i sted Food Management System, portion control 
became a more significant factor to be studied. The Computer-Assisted 
Food Management System would include: 

1. Inventory Control System 

2. Food Cost Accounting System 

3. Food Production and Control System 

4. Analysis of Patient Nutrient Intake 

5. Menu Planning by Computer 

The above system is based on standardized menus which are selected by 
the computer in a least cost manner while maintaining a designated vari- 
ety in menu offerings. A linear programming model is used to accomplish 
this. This system is connected to inventory (automatically signals for 
purchases based on number and type of i ssues) and the cost accounting 
system. It then becomes evident that portion control is essential if 
the above system is to operate smoothly and efficiently. 

A high variance in the weights of some menu items existed during an 
initial data collection period. The problem concerned items where there 
was a choice of using scoops or spoons for portioning. The proper use 
of scoops can eliminate shortages, prevent leftovers, and save time and 
money. However, the argument against scoops has been that scoops make 
the food look too institutionalized, a quality problem of appearance. 
Further investigation into the problem for possible solutions, including 
using automatic portioning equipment, should be conducted. 

X and R charts are going to be utilized on selected menus where 
problems of portioning exist. It is anticipated that they will not be 
necessary for a particular menu after the charts are brought into sta- 
tistical control. It should be noted that control, charts are very good 
tools for studying the variance of a process. They arc also an 



TASTE PANEL EVALUATION 

RATING SCALE: 

PRODUCT 5-like extremely; 4-like moderately; 

CYCLE WEEK 3-neither like or dislike; 2-dislike 

DATE moderately; 1-dislike extremely 

OVERALL APPEARANCE 1 2 3 4" 5 

Circle descriptive factors: 

Correct for product Messy 

Appetizing Signs of Deterioration 

Colorful 


OVERALL AROMA 

Circle descriptive factors: 
Correct for product 
Aromatic 
Odorless 
OVERALL TASTE 

Circle descriptive factors: 
Correct for product 
Spicy 
Strong 
Tangy 

OVERALL TEXTURE OR CONSISTENCY 
Circle descriptive factors: 
Correct for product 
Grainy 
Smooth 
Chewy 
Lumpy 


1 2 3 4 5 

Burnt or Scorched 

Dull 

Putrid 

1 2 3 4 5 

Too sweet 
Too sour 
Too salty 
Flat or Bland 

1 2 3 4 5 

Tough 

Tender 

Thick 

Thin 


OVERALL ACCEPTABILITY : ACCEPTABLE UNACCEPTABLE 

COMMENTS : 


FIGURE 1. Taste Panel Evaluation Sheet 


excellent tool for involving employees in quality control because vari- 
ance can be seen by employees and feedback is immediate. Therefore, 
they will be used at different times for study or improvement projects. 
They may, in the future, find continuous use if new automatic portioning 
equipment is implemented. 

A Patient Pood Service Survey Questionnaire has been developed by 
the VA to be distributed to patients. Figure 2 shows a form developed 
by McLaren(3) which has proved to be very useful for patient input. 
Information concerning items missing from trays, quality of items, and 
request for menu suggestions are solicited using the form. A space also 
is reserved for additional comments. To boost employee morale, favor- 
able comments from patients are brought to their attention. 

SPECIAL PROCESS STUDIES 

This job of quality control is organized to handle special studies. 
For example, a study was performed to determine if one of five bakery 
items was significantly better than the others. Four of the items were 
from different vendors and the remaining item was from the hospital 



COMMUNITY HOSPITAL OF INDIANAPOLIS, INC. 


Time Day Dat_e_ 

PATIENT - FOOD SERVICE SURVEY 


INTRODUCTION : 

This survey is being conducted as part of a continuing effort to provide 
good food service. Your cooperation in completing this questionnaire 
will be most appreciated. If you wish, you may leave your name off, but 
please complete the rest of section 1. 

In section 2 we are interested in learning your general opinion of our 
food service. For each topic, check one square to indicate the phrase 
that best describes your opinion of that aspect of the food service, and 
make any additional comments under each phrase. 

1. Name Room Number Current Diet 

Length of current hospital stay 


SURVEY 

Very good 

Good 

Fair 

Poor 

Very Poor 

Hot food temperature 

□ 

□ 

□ 

□ 

□ 

Cold food temperature 

□ 

□ 

□ 

□ 

□ 

Attractiveness of 
trays served 

□ 

□ 

□ 

□ 

□ 

Selection and variety 
of food 

n 

□ 

□ 

□ 

□ 

Cleanliness of dishes, 
utensils, and trays 

□ 

n 

□ 

□ 

□ 

Handling ease of 
utensils and dishes 

□ 

□ 

□ 

□ 

□ 

Quantity of food 

□ 

n 

□ 

□ 

□ 

Quality of service 

□ 

□ 

□ 

□ 

□ 

Quality of food 

□ 

□ 

□ 

□ 

□ 

Have you received all 

of the items 

marked 

on your 

menu? 

Yes 


No If no, what was missed and about how often?_ 


4. Has someone from dietary visited you? Yes No 

5. What foods would you like to see on the menu that do not appear now? 


6. What foods that do appear would you like more frequently? 


7. What foods that do appear would you like less frequently? 


8. ADDITIONAL COMMENTS: 


FIGURE 2. Patient Food Service Survey 



bakery. A ranked sum analysis of variance was suggested as the means of 
analysis. Many continual problems such as pre-port ioned frozen meat 
versus having a meat shop fall into this job function. Special process 
studies are especially pertinent to the VA in relation to quality of 
convenience products since the VA is interested in obtaining and main- 
taining quality convenience products for patient use. 

THE FOOD QUALITY CONTROL ORGANIZATION 

Before the four jobs of food quality control can be accomplished in 
an integrated manner, an organizational structure needs to be meshed 
into the overall food production organization. The proposed organiza- 
tion should be a staff function reporting directly to the Chief of 
Dietetic Service. This is sho\vn in Figure 3. 

Unlike large organizations with at least one employee in each job 
of quality control, only two employees will be required. Since dietetic 
personnel presently spend time on quality functions, it is anticipated 
that, by reassignment of functions, only one additional employee will be 
needed--the quality manager. It is also expected that the savings re- 
sulting from implementing the jobs of quality control will more than 
offset the expense of the quality manager. 

The success or failure of the quality control organization depends 
largely on the quality control manager. The most important qualifica- 
tion for the quality control manager is that he or she be a good admin- 
istrator. A registered dietitian with a Masters Degree related to man- 
agement, advanced training in quality control administration or experi- 
ence in a quality control program would be preferred. A basic under- 
standing of statistics and sampling tables is a must. A Certified 
Quality Engineer would also be a possibility. Another possibility is 
that a current staff dietitian working in administration be trained to 
develop and carry out a quality control program. The primary respon- 
sibilities of the Quality Control Dietitian would be the development of 
a quality control program that fits the needs of the institution, 
development, of procedures for inspection techniques and data collection, 
determining statistical methods and sampling tables that are to be used, 
analyze data, follow up, regular quality audits, and total communication 
with the Chief Dietitian. 

The quality control technician should be a person with a B.S. 
degree and the ability to work well with the food production personnel. 
The primary responsibility of the technician would be to relieve the 
Quality Control Dietitian of routine quality control tasks, for example, 
the daily taste panels held in production, data collection and mainte- 
nance of control charts. 
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A WEIBULL SHELF-LIFE MODEL FOR PHARMACEUTICALS 


John E. Ogden, Quality Consultant 
Abbott Laboratories 
North Chicago, Illinois 60064 


INTRODUCTION 

Much new statistical technology has been developed in the aerospace and defense 
industries to meet the conflicting demands and constraints of cost, schedule, product 
reliability, safety, and performance. This paper deals with an application of the 
Weibull distribution which has found utility in the field of product reliability to 
the problem of establishing expiration dates for pharmaceutical and other health care 
products . 


EXPIRATION DATING 

Expiration or shelf-life limits are placed on pharmaceutical products to protect 
the user from low and sometimes high potency or concentrations of ingredients. The 
active ingredients may be degraded or become unavailable because of interactions with 
other ingredients or even the container itself. On the other hand, an ingredient may 
become more concentrated because of the evaporation or sublimation of other components. 

Sometimes it is possible to include an excess of an ingredient at the time of 
manufacture to compensate for the aging process. However, cost and the safe limits of 
concentration are limiting factors. Substituents, or materials having equivalent 
biological performance but which tend to preserve the active ingredient, are sometimes 
used to obtain longer shelf life. For example, a glycerine having a lower water con- 
tent than the U.S.P. glycerine might be used to reduce the degradation of an ingredient 
which oxidizes in the presence of water. 

The mathematics of establishing shelf-life values based on experimental data have 

(1 2 ) 

been given thorough exposure in the literature. * A great deal of thought and 
investigation has been applied to problems of setting shelf life. Whole symposiums 

(3) 

have been devoted to the general subject. 

TECHNOLOGY TRANSFER 

Statistics is far from being a unified field. Practical statisticians tend to 
become heavily involved in the particular types of problems and the language of the 
field to which they apply their statistics. It becomes difficult to read technical 
literature written by other statisticians or workers in different fields of application 
because of language differences and different constraints on the application of even 
standard statistical methods. 

For example, for those involved in the application of statistics to the control of 
quality in a machine shop, the word "tolerance" is used to connote a set of limits for 
a physical dimension such as the length of a shaft. Inspection data might give a 
distribution of shaft lengths for a sample of shafts taken from a production lot. The 
tolerance limits are fixed boundaries. On the other hand, in the application of sta- 
tistics to biology, one might administer a dose of digitalis to a group of laboratory 
animals. The dosage would be increased until the animals' hearts stop beating. The 
amount or dose required to stop the heart is not the same for each animal. Thus there 
is a distribution of "tolerance" among the animals. Here the word tolerance is used 
to connote a distributed variable rather than a fixed boundary. 

Communication problems reduce the opportunity to share ideas developed in one 
field of application of statistics with statisticians working in other fields. 
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THE EXPONENTIAL DECAY CURVE 


The application of statistics to the field of reliability engineering has led to 
a number of interesting and powerful techniques. Of course, not all of these will find 
application in the pharmaceutical manufacturing field. The constraints on problems are 
different, and the ability to control various parameters is different. However, there 
appear to be some common types of problems and conditions which lead to the possibility 
of technology transfer. 

One of the more common techniques used for setting shelf-life limits on pharma- 
ceuticals and other health-care products employs the assumption that the concentration 
of an ingredient decreases exponentially with time. This kind of decay function (or 
model) is found to apply in many natural and man-made processes. For example, in the 
electronics field, an electrical charge stored on a capacitor leaks off through a shunt 
resistor giving an exponential decay to the voltage across the capacitor. 

In general, this model can be used to describe any process in which there is a 
fixed rate of decay. That is, a fixed fraction of the substance which remains at any 
given time will be lost in the next time increment. The form of the exponential is 
shown in Figure 1. 


R 


1.0 


0.0 


0.0 Time, t 



R = Fraction of original 

material surviving at t. 
r = Erosion rate, 
t » Elapsed time, 
e = 2.718, base of natural logs. 


FIGURE 1, The exponential decay model. 


This type of curve is common to both the pharmaceutical and the reliability 
fields. In the reliability field, it is often found that the erosion rate, r, which 
is called the failure rate, is not a function of time. If a fixed time interval of 
testing is used, the number of failures observed will tend to follow a Poisson distri- 
bution. See Figure 2. 


PROBABILITY OF 
FAILURE, OR 
RELATIVE FAILURE 
FREQUENCY 

f(t) 



NUMBER OF FAILURES IN t - 100 hours, c 


FIGURE 2. Poisson distribution of failures for fixed 100-hour 

test periods and a failure rate of 0.018 failures/hour. 


A quick look at the equation for relative failure frequency in Figure 2 shows 
that it becomes the reliability equation of Figure 1 when the number of failures, c, is 
set equal to zero; that is, the probability of surviving t hours is the same as the 
probability of having zero failures. 


f(t) 


. (rt) c 


(rt) 


0 o-rt 


0 ! 


R, for c = 0 
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One application of the failure distribution type of model to pharmaceuticals was 


made by Berrittoni. 


(4) 


He modeled the number of drug failures as a function of age. 


Another way to examine pharmaceuticals arises out of the data which result from 
"stability testing." Samples of product lots are placed in storage under "typical" 
conditions. Periodically, samples are withdrawn from storage and assayed. The statis- 
tic typically reported is relative potency or relative concentration of some given 
ingredient. Thus there is no "failure data" as such. One is really tracking the shape 
of the decay curve of Figure 1. 


UNSTABLE DECAY AND THE WEIBULL MODEL 


Sometimes it is found that the decay curve seems to go through a change in life. 
That is, the erosion constant seems to be a function of time, or it may suddenly change 
its value after some period of time. From a diagnostic standpoint, it would be nice to 
know more about the nature of the change in the erosion constant. There is a type of 
erosion model which has a shape parameter which could give information on how the decay 
rate is changing. It is called the Weibull model. 

In the early 1900’s, Fisher and Tippett's investigation of the distribution of the 
largest and smallest observations in samples from a normal distribution led to the 

development of the extreme value distribution.*^ Just before World War II, Waloddi 
Weibull rediscovered the extreme value distribution from a different approach. He 
applied the distribution model to problems involving fatigue and failure of materials. 
Shortly after the war, when the new breed of reliability engineers were searching for 
mathematical models for the equipment failures they were studying, Weibull’ s model 

seemed to fill the bill. It could handle the three life patterns that had been 
observed: burn-in (decreasing failure rate with time), useful life (constant failure 
rate) , and wear-out (rising failure rate) . 

In essence, the Weibull model is a double exponential with three parameters: 
a shape parameter, a scale parameter, and a location parameter. The shape parameter 
(6), allows fitting to a decreasing (3<1), constant (3=1), or increasing (3>1) erosion 
rate. The scale parameter (a) locates the 63% point on the cumulative distribution. 

The location parameter (y) allows one to "start" life at any point in the age axis. 

The equation for the Weibull model is: 


R = 


-(t - y)‘ 

^ n-J— 


( 1 ) 


Where: a - Scale parameter. 

3 = Shape parameter. 

Y = Location parameter, usually set to zero, 
t = Cumulative unit age. 

R = Fraction of original population surviving at t. 


For many reliability applications, the location parameter is zero; and the Weibull 
model reduces to a two-parameter form. 

The shape parameter (3) appears to have value in the analysis of data resulting 
from pharmaceutical stability or shelf-life studies. A value of 3 less than one 
indicates a decreasing erosion rate. This might be due to the exhaustion of a reagent 
that was causing erosion of the ingredient being studied. On the other hand, a 3=1 
value would indicate the type of kinetics commonly assumed; i.e., a constant erosion 
rate. 



The mechanics of erosion have been explored for some types of pharmaceuticals and 

f 7 8) 

give explanations for some of the observed data.' * J For example, Hammett’s linear 
free energy relation of sigma-rho, which allows one to estimate the effects on the 
stability of a compound when substituents are used, is in effect a shifting of the 
scale factor, (a), in the Weibull model. 


log (k/k Q ) = ap (2) 

Where: k = New erosion rate with substituent. 
k Q = Original erosion rate. 

a - Substituent constant which depends solely on the 
substituent 

p * Reaction constant, constant for all substituents. 
Equation (2) may be rewritten: 


Let 


k = k e 
o 


2.303 op 


i/i i/i 2.303 ap 
= 1/k = 1/k e 


Substitution in equation (1) gives 

-It - v ) 6 

R = a a = 


an equation of the Weibull form: 


-tk 


-t/a 


1, y = 0 


Thus, the Weibull gives two parameters, (3 and a, which separate out the two 
primary characteristics of erosion. This not only simplifies the statistical problems 
by reducing the number of parameters that must be estimated from the test data., but it 
reduces the amount of data required to detect problems in rate kinetics. By separating 
life "problems" into two characteristics which relate to the basic mechanics of decay, 
it is also much easier to obtain clues as to the physical cause of problems. 

The two primary characteristics of erosion, 6 and a, answer the questions. Is the 
erosion rate independent of time, or does it increase or decrease with time? and Is 
there a difference in the basic erosion rate for two materials even if both have 
erosion rates that change with time? 


WEIBULL PAPER 

Graph papers have been developed to allow plotting the cumulative experience 
against time. By fitting a straight line to the data, an estimate of the size of the 
two parameters can be obtained. If a break or change of slope occurs, the location 
parameters for the two life patterns can be estimated; and the two curves, thus 
separated, give rise to different shape and scale parameters for the two life segments. 

As with any attempt to estimate parameters from sampled data, one must be careful 
not to place undue confidence in them. Formal statistical techniques exist for 
placing confidence limits on various estimates. However, the value of graphic methods 
is that they give clues to the causes of the behavior observed with a minimum of cal- 
culation. Obviously, more formal methods are desirable when the consequences of being 
misled are serious. 

An example of stability data plotted on Weibull paper is shown in Figure 3. The 
basic data are shown in Table I. These data were "contrived" or generated by simula- 
tion methods so that they do not represent any actual product. This was done not only 
to preserve proprietary actual stability data, but because by knowing for certain what 
factors generated the data, it is easier to illustrate how the Weibull parameters act 
as pointers to various conditions. 




The data in Table I were generated by drawing random samples from a normal 
distribution which simulates assay and lot-to-lot variability in the concentration 
of the active ingredient, Natural logs of the assay results were taken, and the means 
of the log-concentration data are shown in the table. Ten time periods are shown. 

The time intervals are assumed to be equal. The erosion rate was assumed to be -.02007. 
This should theoretically cause a reduction to 90% of label claim at the end of ten 
time periods. The assumed concentration at release from the factory was 55 mg/ml with 
a 50 mg/ml label claim. The variability of assay and lot-to-lot differences was 
assumed to give a standard deviation of 2 mg/ml. The sample size used to generate 
each mean was 6. 


TAB LE X SIMULATED STABILITY DATA 


TIME 0123456789 10 

f C 55.000 53.907 52.836 51.787 50.758 49.749 48.761 47.792 46.843 45.912 45.000 

THEORY < R 1.000 0.9801 0.9607 0.9416 0.9229 0.9045 0.8866 0.8690 0.8517 0.8348 0.8182 

^ l-R 0.000 0.0199 0.0393 0.0584 0.0771 0.0955 0.1134 0.1310 0.1483 0.1652 0.1818 


t xl 53.96 54.05 52.78 53.89 48.24 51.41 47.87 48.05 47.01 41,90 48.08 

SAMPLED x2 56.59 52.51 54.68 55.25 51.63 49.69 49.61 47.97 48.78 49.02 46.13 

DATA ( x3 57.09 53.30 56.03 49.24 55.64 50.62 47.59 47.52 46.48 48.32 43.50 

] x4 56.73 50.99 50.71 54.51 50.16 45.22 47.09 47.60 44.87 46.10 46.02 

x5 54.78 57.40 52.56 51.69 48.13 52.31 50.77 47.53 44.87 46.08 43.34 

1x6 53.42 54.94 52.99 49.73 50.24 50.10 51.35 47.07 47.69 46.84 45.29 


u = Avg ln(x) 4.0148 3.9852 3.9753 3.9576 3.9242 3.9088 3.8922 3.8633 3.8418 3.8355 3.8147 

S = ln* 1 (C> 55.215* 53.793 53.265 52.333 50.621 49.839 49.019 47.622 46.609 46.617 45.363 

R - E / c 0 0.9743 0.9647 0.9478 0.9167 0.9027 0.8878 0.8625 0.8442 0.8389 0.8216 

l-R 0.0257 0.0353 0.0522 0.0833 0.0973 0.1122 0.1375 0.1558 0.1611 0.1784 

NOTES: 

* 55,215 represents the Intercept (t=0) from the least squares line fitted to the averages of the ln(x) 
data points. This gives the "best estimate" of the initial concentration by utilizing the entire data 
set rather than using the t“0 sample. 


The units for C.xi, and c are mg/ml 
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The dashed line from the pivot point drawn at right angles to the fitted line 
passes through the shape parameter estimation scale at a value of 3=1. This indicates 
that the data can be modeled by an exponential curve with a degradation factor which 
does not change with time. This is, of course, in agreement with the theoretical 
function which was used to generate the data. 

The least squares line was fitted to the means of the natural logs of the concen- 
tration data. Any attempts to use the fitted line for purposes of forecasting or any 
of the standard regression analysis techniques rest on the assumption that the fitted 
numbers are normally (or nearly so) distributed. By using the mean of the natural logs, 
the law of the mean assures reasonable normality to the averages of six numbers even 
though the original numbers (natural logs of concentration in this case) are not 
normally distributed. 

The general procedure for plotting Weibull paper has been described by J. N. 

(4) 

Berrittoni, and an interesting method for testing the fit of data to the assumed 

(9) 

model has been suggested by Weibull. 

A second example using simulated data is shown in Figure 4. In this instance, 
a decreasing erosion rate was programmed into the simulation for the first six time 
periods. This was used to simulate a drug with a reagent which gradually exhausts 
itself; and then after the sixth period, the erosion rate becomes uniform. 

The Weibull parameters used in the simulation were: 

A. Life period "A” (first six periods): 8 = 0.6, a = 8.22, y - 0 

B. Life period "B" (after sixth period): 8 = 1.0, a = 16.82, y = 0 

Examination of the plots in Figure 4 gives the following estimates: 

A. Life period "A": 8 = 0.6, a * t® = 34 0,6 = 8.3, y - 0 

B. Life period "B": 8 = 1.0, a = t® - 16, k = a” 1 = .06, y = 0 


*4)-* ESTIMATION WEIBULL PROBABILITY PAPER 

PIVOT POINT p 

\\ R * «*p(-U - »>/*> 

\ \ 




CONCLUSIONS 


The Weibull model provides a convenient means for separating the influences on 
degradation into three parameters. The shape parameter gives an indication of whether 
the erosion rate is increasing, constant, or decreasing with age. The scale parameter 
allows comparing two or more lots or types of material with respect to the basic rate 
of erosion independently of whether the rate is constant or not. The location parameter 
allows one to locate changes of life pattern and assign a point of origin to erosion. 

Fitting data to Weibull paper is not an exact method of fit. However, it provides 
a rather quick and illuminating way of examining life patterns. 

The Weibull model appears to have many diverse applications, and it merits 
consideration by investigators who are looking for a life model which has parameters 
that relate to the mechanical conditions which cause life patterns to exist. 
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CAN CONCEPT-TO-CONSUMER SLIPPAGE BE PREVENTED? 


George J. Lutz 

Manager, Consumer Technical Services 
Maxwell House Division 
General Foods Corporation 
White Plains, New York 10625 


Many new products fail to make the transition from promising 
concept to successful marketed entity. The specific causes of 
failure can be any of a number of factors. However, top management 
may not be aware of how a strong quality assurance system might 
improve its success ratio. 

The quality assurance role centers on technical slippage between 
concept and marketed product. The answers we hope to develop to the 
question of can this slippage be prevented will be particularly 
suitable for the developers of consumer products. If we were to 
include the space industry, for example, in the scope of this paper, 

I believe we would find that their top management was interested in, 
knowledgable about, and involved in the support of a strong quality 
assurance system. This, because almost their entire effort is 
devoted to the effective application of existing technology. The 
technical issues, rather than marketing, sales, profit or budget, 
dominate management's time and energies. But in the retail consumer 
industries we have the other end of the applied technology spectrum. 
Here the technical issues tend to receive less of top management’s 
time. Management tends to assume that an adequate organization is in 
place and that through it the marketing concept will become the 
desired product — and will be available at the time and for the 
price prescribed by the marketing plan. 

There is evidence, however, that reliance on the existing tech- 
nical organization may not be enough. Even when the separate 
technical functions are strong, they may not be effective in chan- 
neling their strength toward the mutual objective. Even when the new 
product technical effort is strengthened by the clear assignment of 
centralized leadership, it is hampered by a limited time frame so that 
it is not possible, in the time available, to overcome long-standing 
obstacles to effective communication across functional lines. 

Communication is viewed by some as just one chapter in the book 
of management. By others it is viewed as the whole book. This 
presentation leans heavily toward the second viewpoint. It says, in 
effect, that the management book on new product introduction is a 
book on effective communications — primarily effective communication 
of the technical information necessary for the successful manufacture 
of the new product. The vital core of that communication network is 
a strong quality assurance system. Of course, there are also many 
organizational, financial, and decision-making pitfalls along the way; 
but top management appears to be aware of these and capable of dealing 
with them. This may not be so true of the technical pitfalls. 

THE PRACTICAL REWARDS 

The need then is to bring management "up to speed" on the role 
they can and must play to insure that their overall plan for a new 
product introduction includes a strong quality assurance system. Of 
course, this is a lot easier said than done? but, to avoid losing 
their interest while we get at the problem, we might hold their atten- 
tion by suggesting that there are very practical rewards in store for 
a manager with this vision. The major rewards are more successes, 
earlier successes, and increased investment confidence in new 
products. 
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To spell out a bit how a sound quality assurance system can 
dangle such rewards before a manager: 

1) More successes : While the goal is no slippage between 
promising concept and product in full-scale production/ the realistic 
expectation is minimum slippage. Fewer failures, of either a 
temporary nature or a permanent one, can be expected when effort is 
organized to prevent any failures. 

2) Earlier successes : This applies both to reaching satisfac- 
tory full-scale production earlier and to getting to payback point 
earlier. Time won't be wasted by the situation that occurs — not 
uncommonly — when a sound q.a. system is not in place and the product 
designed by Research is inadvertently redesigned by Engineering 
because of faulty communication and then redesigned by Manufacturing 
because their needs were not considered in the early phases. It has 
also been demonstrated that higher yields and greater productivity 
are the natural outgrowth of a sound q.a. system — leading to 
meeting economic criteria sooner. 

3) Increased investment confidence : This is really an outgrowth 
of the first two rewards. Some new products fail not because the 
concept could not eventually be made a reality, but because it took 
too long and required too much investment up front to bring this 
about. When the success ratio goes up and development time is 
reduced, top management has two hole cards that give it the confidence 
to put up the ante and stay in the game. 

SOME PRINCIPLES OF A Q.A. SYSTEM 

Because the quality assurance system is, in effect, the organized 
use of technical procedures and techniques, it is difficult to 
communicate to top management people. They are more interested in, 
and would better understand, the principles on which the q.a. system 
is based. Three of these should be mentioned: 

First , quality is not an abstract idea but a collection of 
limiting factors (specifications) that set the yield/cost potential of 
a process. 

Second , a quality assurance system is an interdepartmental 
c ommun ic a t i o n system. 

Third , the end product of the system is a process and product so 
well-defined that it can be readily maintained and further developed. 

Let's develop each of these principles a little further. 

Quality Is Not an Abstract Idea 

The first step in the conversion from new product concept to 
continuous commercial production is the struggle to describe 
accurately in technical terms (specifications) all of those product 
properties which completely define it (i.e. define its quality). 
Management should not underestimate the difficulty of this first step 
or the difficulty communicating these specifications in a way that 
gives them permanence and immutability. No, quality is not an 
abstract idea. It is the concrete foundation. 

An adequate quality assurance system can maintain the written 
specification to whatever extent it is developed by research and 
engineering. There need be no slippage on this first step. 

Recognize, however, that each technical specification is a limit 
put on the manufacturing department. It will eventually, if not 
immediately, be an obstacle to manufacturing's ever-present objectives 
of lower cost operations, whether in productivity, yield or manning. 
The quality assurance system must be strong enough to protect the 
specification until such time as management chooses to change that 
specification. 


Let's go on to the second principle. 

Inter-departmental Communications System 

The management of every manufacturing company has developed an 
organization which it believes can carry out the tasks at hand. 
Responsibilities are assigned; and we can assume each department 
clearly understands the limits of its responsibility and therefore — 
at least in general terms — the scope of its authority. However, 
unless the boundaries of these areas of authority are re-inforced or 
clarified within the framework of a quality assurance system, com- 
munications between departments becomes difficult and important 
technical issues frequently become jurisdictional issues. It is not 
uncommon under these conditions for the specifications to begin to 
shift. After all, marketing and manufacturing can't wait for the 
jurisdictional dispute to be resolved. So, slippage can occur by 
default. Management's commitment to the quality assurance system can 
give the necessary traction at this slippery point by establishing 
formal procedures for dealing with the "gray" areas. 

And this leads us to the third principle that management should 
understand. 

A Well-Defined Product 

Our first principle , stressing the importance of specifications, 
looked at quality assurance's defensive contribution — that is, how 
a strong q.a. system can keep specifications from being weakened or 
changed. But it would be desirable to have more from a quality 
assurance system than just barriers to slippage. The system should 
make a positive contribution — initially to the most accurate conver- 
sion from concept to full-scale production; and then, beyond that, to 
the evolving product after it is in full-scale production. 

A sound quality assurance system does make this positive contribu- 
tion — through the formal mechanism it establishes. This mechanism 
is the system characteristic most important to the development of the 
accurately defined product. It is the basis for establishing consis- 
tent, functional specifications for both product and process. Because 
a well-defined product can more clearly be understood by both the 
technical and the manufacturing communities, such a product should, in 
turn, lead to the least-cost process at design quality — the one that 
gives management maximum flexibility and leverage in the market place. 

ELEMENTS OF A Q.A. SYSTEM 

Having said that management would not be "turned on" by the 
details of the quality assurance system, we must also say that it 
would certainly be "turned off" if we were not prepared to describe 
in some detail what the system is. 

Let's spend a few minutes on what a viable quality assurance 
system looks like. In our experience there are half a dozen elements 
that must be combined in a formal, integrated way in such a system. 

We are not saying that this is a new view. It's an old one. In a 
way, we are trying to get into the mind of a craftsman to see how he 
works, to try to de-mystify what he does. If we could do this, we 
might see that the complex of people and equipment that make up 
today's manufacturing process is the embodiment of yesterday's crafts- 
man. That's an oversimplification, of course, because size and 
sophistication add their own dimensions to the problem of quality 
control. But let's stay with the analogy because it does help us to 
make important points about the elements of the quality assurance 
system. 

What the successful craftsman has or does comes to at least six 
basic elements. 1) He has access to all of the important facts 
about his skill. 2) He has acquired his skill through apprenticeship 
and training. 3) He follows an organized work plan or approach that 
has been shown to be reliable. 4) He periodically judges his perfor- 
mance against established standards of quality. 5) He makes sure his 
left hand knows what his right hand is doing. 6) When needed, he 



takes his corrective action in a disciplined way based on the 
accumulated experience of his trade. 

We can probably all agree that in general terms that sounds right. 
But we should get more explicit about what this means in a complex 
manufacturing unit. 

Knowledge 

The most important single element for manufacturing a product 
that meets the design criteria is the accumulation, in usable form, of 
the best available knowledge — knowledge that issues from, and is 
binding on, all groups concerned. We've mentioned the importance of 
specifications and test methods, but that is only the beginning. All 
raw materials must be cleared; vendors must be reliable and must be 
monitored; facilities must be adequate; process equipment must be 
reliable and capable of producing design quality product. 

Since many departments and technical groups can be involved here, 
all must feel they are contributors to the required body of knowledge 
and also must feel obliged to accept the contributions of others — at 
least until differences can be resolved. All of these people are 
spokes of the wheel anchored to the documents that are at the hub. 

When management commits its support to the quality assurance system, 
it is committing support to whatever extra effort is required to 
develop and maintain the complete set of documents. Most technical 
groups find the chore of putting findings in writing rather anti- 
climatic and unrewarding compared to the joy of reaching those findings. 
Even more difficult for most is the struggle for consensus on those 
issues about which good men can technically hold differing points of 
view. Since the documents — the records of accumulated knowledge — 
are the basis for accurate, efficient communication and activity, their 
importance cannot be over-emphasized. The resources needed for their 
development and maintenance should be identified and funded up front. 
Then the leadership must be found to organize the people and establish 
the procedures that will overcome the obstacles. 

Skill Development 

All of the knowledge accumulated in the documents is of little 
use if it is not made available to the people — operators, engineers, 
researchers — who can use it. Particularly important is establishing 
adequate training programs for process operators. A successful 
training program is the one that gives the man with his hand on the 
throttle and valves all the guidance and information he needs to 
prevent the defect from occurring. Any compromise with the full 
training effort needed for this just increases the risk — and 
probability — of defects. 

In most manufacturing processes, it is not art that is required 
of an operator, but skill. It is the training program that helps the 
individual make the transition from artist to skilled operator, 
someone who reliably turns out design quality. 

From another point of view, the lack of sound training also limits 
the contribution the operator can make, as one of the spokes of the 
wheel, in feeding valuable information back to the document point at 
the hub. We have found that increased operator involvement generates 
useful ideas for improved process control. It also improves individual 
attitudes and helps standardize shift-to-shif t operations. 

A Plan 

This element has to do with the thorough-going, organized approach 
to the production task at hand. , Every step in the production sequence 
must be set down as it is in the mind of the craftsman, and then 
individuals must clearly be assigned their part in the integrated 
whole of the process. In other words, we have to overcome the problem 
we created by fractionating the craftsman's role into many parts. 
Nothing short of a detailed written plan with clear-cut assignments 
can make this happen. 
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At General Foods, we have found that this written plan approach 
makes it possible to identify and eliminate confusion about both 
individual and department responsibility. Early on, it was particu- 
larly helpful in clearly defining the operating department's 
responsibility for quality control. 

The Quality Check 

As part "of the plan, each operator in the processing department 
should make as many quality measurements as required to keep his part 
of the process and product on target. But beyond that, we need the 
equivalent of the single pair of eyes of the craftsman who can see the 
total finished product and judge it independently and objectively. In 
a system, this independent view comes from the separate department 
(Quality Assurance) that maintains the skills for such overall product 
assessment. 

Left Hand Tells the Righ t 

Nothing underscores the difference between the individual crafts- 
man and his many replacements in modern industry than the simple 
problem of the left hand (in one department) letting the right hand 
(in another department) know what is going on. So in industry we have 
to form the equivalent of the human network of messages that go from 
the craftsman's brain to his arm, leg, or finger with directions to 
take the correct action. The data must flow as freely and reliably as 
it does in the biological model. Data must arrive in the shortest 
time possible, because early clues help prevent drift and defects. 

Experienced Handli ng 

We said "help prevent" because the transmittal of timely, accurate 
data is, in most cases, not enough. Raw data is not as useful or 
powerful as analyzed data. So a final element deals with extracting 
from the data as much useful information as possible to make the 
process control decisions the correct ones. The more help an operator 
or his supervisor can get in the form of simplified data analysis 
techniques the better. Then the control actions can be specified and 
following them can be made a firm requirement. Where not specified, 
the decisions should be escalated to the next level of supervision. 

This is the equivalent of thinking about it a bit before taking actions 
which are risky because their effects are not proven. 

Those are the elements of a sound quality assurance system. It 
depends at bedrock on the early and continuing input of research and 
engineering, but the effort for start-up and maintenance of the system 
falls to manufacturing. All three departments stand to benefit along 
with the company as a whole when a new product is close to the 
original concept. Manufacturing, in particular, stands to benefit 
longer-term through higher productivity, lower re-work of defects, and 
better communications with the technical community for process 
improvements . 


WHAT CAN MANAGEMENT DO? 

If we have held management's attention to this point with our 
promise of rewards, we might find them asking, "Exactly what is it 
you want us to do?" For starters we might suggest that they recognize 
that the quality assurance system is not the province of one depart- 
ment. Only with this recognition will come the multi- function 
accountability to go with the shared responsibilities. We should tell 
them to forget the stereotype of the Quality Assurance Department 
doing its own thing, i.e., keeping the boys honest (but possibly only 
after the value of the goods has inadvertently been eroded away) . 

They should see the system as the thin^ . The Q.A. Department has a 
prime role in coordinating and monitoring system effectiveness, but 
the key to successful application lies elsewhere. . . 

- In Marketing Research, for a clear and accurate measure of the 
importance of quality factors to the consumer. 

- In Technical Research, for the accurate product design through 
a proven operational process which they put into the documents. 


- In Engineering, for their efforts to convert Research develop- 
ment experience accurately through processing equipment which 
permits control of the essential variables. 

- In Manufacturing, for the staffing, initial training and persis- 
tence in use of the system. 

Each of these major participants in the system must be recognized 
(and rewarded) for his part in making the system work. Of course, the 
objective of getting the new product on stream is still primary; but 
we go so far as to say that achievement of that objective depends on 
the successful application of the system. Furthermore, without the 
system, accountability is frequently difficult to determine; and you 
end up without the ability to pinpoint weaknesses and then do some- 
thing about them. 

In addition, we should ask management to be anticipators. They 
should not wait for the new product to be looming on the horizon 
before they call the q.a. accountability roster. In other words, the 
quality assurance system should be in place in the existing organiza- 
tion before the new product gets much beyond the concept stage. Then 
the various functions are comfortable with their roles and responsibil- 
ities. They will have seen them work for existing processes and 
products. They can be held accountable. 

CONCLUSION 

The top management of companies which rely on new product entries 
for growth (or survival) should be aware of the importance of a strong 
quality assurance system, should know what such a system consists of, 
and should do their part to make such a system a reality. 

- They can emphasize the need to develop and adopt a quality 
assurance policy and procedure that encompasses all of the 
departments that can affect quality — so that the communica- 
tions network is in place. 

- They can insist that the system be in place and functioning 
before the new product leaves the development laboratory — so 
that everyone is familiar with the system's workings. 

- They can see that compensation/rewards are based on effective 
performance within and in support of the system. 

The motivation to management action, of course, is the awareness 
of the possible rewards in new product conversions — more successes, 
earlier successes. 
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USE OF REGRESSION ANALYSIS IN PACKAGING INSPECTION 


Marvin Sirota, Corporate Director of Quality Assurance 
Helene Curtis Industries, Inc., Chicago, Illinois 

Michael Kamowski, Inspection Manager 
Helene Curtis Industries, Inc., Chicago, Illinois 

Mark Wozniak, Quality Assurance Chemist 
Helene Curtis Industries, Inc., Chicago, Illinois 


INTRODUCTION 

In our operation, we receive annually between 75 and 100 million plastic con- 
tainers for our many products. Each shipment is inspected by our Receiving Inspection 
Department for package conformity to specifications. A complete analysis can take 
from 45 to 90 minutes. 

It was our feeling that the data generated by Inspection should be utilized in 
more ways than "go - no go". 

A project was initiated to establish whether significant correlation exists 
between the weight and overflow capacities of plastic containers from a given mold. 
This information was evaluated for better utilization of inspection time, informing 
vendors of their performance, and improving line efficiencies through the reduction 
of product give away. 


PROJECT 

We annually use in excess of 75 million plastic containers for product packaging. 
These containers are received in shipments ranging in size from 20,000 to 250,000 
pieces. 

The sampling procedure employed is to select a minimum of ten cases of bottles 
from each shipment. In the selection process an effort is made to insure that each 
vendor manufacturing date and shift are represented. Each of the cases in the sample 
is opened and 20 bottles are inspected for manufacturing flaws. A representative 
from each cavity contained in the case is taken and the date and shift information is 
recorded on the bottle. These samples are taken to the laboratory for further 
analysis. 

One of the measurements made is the determination of overflow capacity. Overflow 
capacity is important because it is through the proper establishment and enforcement 
of specifications that we control our solution usage. The label claim is the basis 
for establishing specifications for minimum overflow capacity. Allowances for head 
space and filling equipment variations are taken into consideration in arriving at the 
minimum overflow capacity. Target and maximum specification values are then obtained. 

Each bottle brought to the laboratory for analysis is first weighed on an 
analytical balance. The weight data is recorded. The bottles are then filled with 
water (68 - 70°F) to the overflow of the bottle. The filled bottle is weighed and the 
empty weight is deducted to obtain the overflow capacity of the bottle. 

General observation over the years had indicated an inverse relationship between 
bottle weight and overflow capacity. It was decided to examine this relationship and 
possibly exploit it as a time saving device in our inspection of plastic bottles. 

ASSUMPTIONS AND METHODS UTILIZED 

The plan was to examine the relationship of bottle gram weight and overflow 
capacity through the use of the Pearson Product -Moment Coefficient of Correlation. 
Several assumptions are implicit in our choice of methods. 

1) Plastic bottles, in general, will increase in volume capacity as the weight of 
the bottle decreases, all other factors being held constant. 
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2) Bottles used in study were first visually inspected for physical defects. 

Defective containers were not used in the evaluation. 

Two hundred ordered pairs of data (weight and overflow capacity) were randomly 
selected from one year's accumulated measurements. These ordered pairs were plotted 
on a correlation chart (Exhibit I). On each axis intervals were selected to yield a 
maximum dispersion of values over the chart. The frequency of measurements in each 
interval was then tallied. Calculations to obtain the values fy', (fy 1 ) 2 , fx' and 
(fx 1 ) 2 were performed. 

The value x'y' was calculated in a number of steps. First, the frequency of 
occurence in a given square was multiplied by the value appearing in the upper right 
hand corner of that square. Second, positive and negative totals are obtained 
separately across the rows and entered in the appropriate column. The value 2 x 'y' 
is obtained. 



The value of major importance is "r", (the Pearson Product -Moment Coefficient 
of Correlation). For our purposes a coefficient of correlation i -0.70 was considered 
significant. 

RESULTS 

For all vendors whose bottles exhibited values of r “5 -0.70 calculations were 
continued in constructing a regression line and determining the standard error of 
the estimate. 

Notation used for construction of regression line and standard error of the 
estimate determination are; 

1) Ajj - assumed mid-point of the distribution of x values, i.e., bottle weight 
where x'= 0. 

2) Ay - assumed mid-point of the distribtion of y values, i.e., bottle over- 
flow capacity where y ,ss 0, 



EXHIBIT I 





Notation Used in Calculation of Pearson Product-Moment Coefficient of Correlation 

x - the independent variable; the gram weight of the bottle, 
y - the dependent variable; the overflow capacity of the bottle. 

N - the sample size, 
f - the frequency. 

x'- the deviation of x values from the assumed mid point of the x distribution, 
y'-the deviation of y values from the assumed mid point of the y distribution. 







7) r - Pearson Product -Moment Coefficient of Correlation. 

8) (T* - the standard deviation of x values (calculated by multiplying 

by i x ). 

9) <Ty " the standard deviation of y values (calculated by multiplying 

' C Tj* by iy) . 

10) Formula of Regression Line: 
y predicted 


( r §- x ) ' ( r % x ) 


11) Formula of Standard Error of the Estimate: 


>y . x - (T" y */ l-t^ 


Calculations for Construction of Graph 


r = -0.807 

x - A x + = 55,5 + U 2 -P . l . 55.5 + 0.145 = 55.645 

N 200 

v = A, + = 854 + = 854 + 4.02 - 858.02 

y T n -r 200 


G; = i x ^ = (1) (1.978) = 1.978 
(J^= iy <T^ = (2) (4.819) = 9.638 

y predicted = ^r x^ - ^r x^ + y 

^ * v 

= /-0.807 x'l - ( -0.807 hQl 55.645 J + 858.02 

V 1.978 / \ 1.978 / 

* -3.932x - 218.807 + 858.020 


= -3.932x - 1076.827 


X 

y predicted 

Sy.x - Gy s/l-1'2 

53 

868,431 

= 9.638 /1.0 - 0.651 

55 

860.567 

- 9.638 (0.591) 

57 

852.703 

= 5,694 

60 

840.907 

95% confidence limit = (Sy, x )(2) 


A graph of the regression line (Exhibit II) was constructed using the above 
calculations with bottle weight as the independent (x) variable and overflow capacity 
as the dependent (y) variable. The specified limits for these values (gram weight 
and overflow capacity), are used to construct a "target" area on the graph. 


The regression line is plotted through this target area. An area of twice the 
standard error of the estimate (2 Sy.x) is constructed above and below the regression 
line. The band described, represents the 95% confidence limits for overflow capacity 
at a given bottle weight. 


Weight control limits were established once the regression line has been plotted 
with 95% confidence limits. The lower weight control limit is established at the 
point of intersection of the lower 95% confidence limit (y predicted = -3.932x + 
1076.827 - 11,388) and the minimum specified overflow capacity (y = 841 cc's). 

Bottles which have a gram weight of 57,08 grams will have overflow capacities higher 
than the specified minimum, 957, of the time. 


The upper control limit is established at the point of intersection of the 
regression line and the maximum specified overflow capacity. Bottles which have a 
gram weight of 53.4 grams will have an overflow capacity within the specified limits. 



50% of the time. All bottles having 
weights within the control limits need not 
be measured for overflow capacity. We can 
be assured that bottles within these 
control limits will meet our minimum 
specified overflow capacity more than 95% 
of the time and will not exceed our maxi- 
mum specified overflow capacity more than 
50% of the time. Using the values for the 
average and standard deviation of weight, 
it can be seen that nearly 70% of the 
bottles received will not require verifi- 
cation of overflow capacity. 

The maintenance of the plan calls for 
the weighing of 20 bottles from every 
shipment received. Those bottles with 
weights outside the control limits must be 
filled and checked. It was also decided 
that audits should be made to verify the 
system. 

The graph can also yield other infor- 
mation. For a given weight, the standard 
error of the estimate can be an indication 
of how much control the molder has over 
the various variables affecting bottle 
weight. These variables would include 
distribution, wall thickness, cycle time, 
heat, etc. All of these variables affect 
the values of overflow capacity slightly. ] 
effect on the overflow capacity should be nc 
estimate can serve as an index of the precis 



WEIGHT IN GRAMS 


f they are under perfect control their 
m-existant. The standard error of the 
ion of the molding process. 


The graphs which are constructed of the regression lines can also serve other 
useful purposes. Though the information is available from other sources, the graph 
serves as an impressive visual aid. Three graphs are used to illustrate this point. 
Exhibit III portrayed excellent control over the molding process. The control 
exhibited over both weight and overflow capacity is both accurate and precise. 


Exhibit IV indicates that the vendor is 
precise but not accurate. In this parti- 
cular instance, repairs were requested on 
the molds to bring the capacity back into 
specification. Exhibit V illustrates 
accuracy in meeting the specification, but 
the precision is not as good as the vendor 
represented by Exhibit III. 

This information can be used both 
internally in vendor evaluation, and in 
discussing with a vendor on improvements. 








CONCLUSION 

This method has proved a reliable time saving device in the inspection of 
plastic bottles. The analysis was performed on all of the vendors of bulk plastic 
containers. By utilizing this method, we recognized substantial time savings in 
Receiving Inspection. 

The use of this analysis has also been helpful in suggesting corrective action 
to our suppliers. 

The process has prevented adverse effects in our manufacturing operation. When 
used in conjunction with a policy of strict enforcement of specified limits, product 
give away is held to a minimum. 


BIBLIOGRAPHY 

Juran, J.M. , Quality Control Handbook , 3rd ed., McGraw -Hill Book Company, New York, 
1974. 

Kazmier, L.J., Statistical Analysis for Business and Economics , 2nd ed, , McGraw-Hill 
Book Company, New York, 1973. 

Mason, R.D., Statistical Techniques in Business and Economics , 3rd ed., Richard D. 
Irwin, Inc., Homewood, Illinois, 1974 


LCS 540:70:428 




ON THE USE OF MULTIVARIATE ANALYSIS: AN OVERVIEW 


Allan R. Sampson 
Abbott Laboratories 


INTRODUCTION 

Statistics can be viewed as the mathematical science that allows inferences 
to be made regarding unknown and underlying structures from observed numerical 
data. For example, based on the data from 500 patients in a clinical drug trial, 
inferences are drawn as to the effect of that drug upon a general population. 

Various specialties within the general area of statistics have developed to meet 
particular sets of needs. Nonparametric statistics, for instance, permits inference 
with less rigorous assumptions on the data structure, while sequential analysis is 
most applicable when the data arises sequentially. Multivariate statistical 
analysis permits the analysis of data where multiple observations are taken on 
each experimental unit, e.g., observations describing a given patient's clinical 
response over time. It should be stressed that these various specialities within 
statistics in no way partition the science of statistics. There is in fact much 
overlap and boundaries are very blurred - nonparametric multivariate statistical 
analysis is an active area of application and research. Many of these areas of 
statistics, such as multivariate analysis have their roots in 19th centry mathematics 
but it is only recently with the popularization of high speed computers, that the 
widespread application of these methodologies is possible. 

It is the purpose of this overview paper to discuss the conceptual basis of 
multivariate statistical analysis with emphasis on examples arising from pharmaceu- 
tical research. Before presenting the actual methodological discussion, we 
consider some examples where a natural approach to viewing the data is a multivariate 
one . 

Chronic Toxici ty Studies 

In order to assess the potential toxicological effect of a given compound, 
long term studies over months or years are done, usually with rats or dogs. Such 
an experiment might yield, for example, weekly body weights over 52 weeks from 400 
rats, 100 each receiving either control, low dose, middle dose or high dose. In 
this case, there are 4 x 100 x 52 = 20,800 observations. Oftentimes data such as 
this is analyzed for each of the 52 weeks separately. However, it seems more 
appropriate to view the 52 data points for each rat as an entity in itself, i.e., 
view the collection of these 52 variates in a multivariate fashion. In multivariate 
analysis one employs these collective entities, thereby making use of all the rat's 
data including the weekly interrelationships and body weight growth patterns over 
time. 

Diagnosis 

For certain syndromes multiple item questionnaires are employed for diagnosis 
and for quantification of the severity of the syndrome (e.g., diagnosing and 
measuring anxiety, hyperkinesis, or alcohol withdrawal). For example, if one has 
a 20 item questionnaire, it is natural to view an individual's response to this 
questionnaire as a collection of 20 data points, rather than deal with each data 
point or questionnaire item separately. One can then study the interrelationships 
among responses; or identify subgroupings or combinations of questionnaires items 
that are more meaningful or useful in diagnosing certain subtypes of the syndrome. 

Drug Evaluation 

In order to compare several drugs for efficacy and safety, there are usually 
multiple variables for measuring efficacy and safety. Standardly, for evaluating 
safety, clinical laboratory tests are employed. It is possible to examine each 
laboratory parameter separately in order to assess relative drug safety, but 
often it is more desirable and meaningful to view the laboratory results collectively 
(multivariately ) or, at least, certain subsets (e.g., those parameters pertaining 
to the liver) collectively. 
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MULTIVARIATE METHODS: BASIC CONCEPTS 
The sample mean and variance, as well as the Gaussian distribution, are 
familiar concepts of univariate statistics, where one deals with a single variable 
at a time. The sample mean measures the center of a sample while the sample 
variance measures the dispersion of a sample. The bell-shaped Gaussian or normal 
distribution is quite often assumed to represent the underlying distribution of the 
data. Many of the usual univariate methods, e.g., t-test, F-test, are based on the 
Gaussian assumption. 


Natural analogues of these concepts exist in the multivariate case. Suppose 
that X^,...,Xq represent the q data points associated with an experimental unit. 
For example, q may be 52 and X X52 represent a given rat's 52 weekly body 

weights. It is convenient to represent these q data points as a vertical array 
and then use the simplified notation X to represent the collection of these q data 
points, where we write 


X = 


The entity X is usually called a data vector of dimension q. If there is a 
sample of size n of such sets, this sample is denoted X^,...,X n . For instance, 
if a 20 item questionnaire Cq = 20) is administered to a sample of 35 individuals 
(n = 35), the data vector is 

w\ 


*1 

x 2 


[_ x 20j 

and the sample is denoted by the 35 data vectors X^,...,X 35 . 


For a given sample of multivariate entities of dimension 
a multivariate sample mean X measuring the center of the data 


q, & 

is defined by 


X = (X X + ... + Xn)/n, 


where these vertical vector arrays are combined by adding corresponding entries and 
then dividing by n. The multivariate analogue of variance is slightly more involved 
and is given by numbers arranged in square array fashion. This square array is 
called the sample variance - covariance matrix, denoted by S, and defined by 

S 11 S 12 s qq 

S 21 s 22 s 2q 


|fqi *** s qqj 

where s^ is the sample variance (unbiased) of the ith entry of X and s^j is 

the sample covariance between the ith and jth entries of X. The sample correlation 

between the ith and jth entries of X is s-jji (s-q^/ 2 Sjjl/^). 


Analogous to the univariate case, there is a multidimensional Gaussian 
or normal distribution. Many of the standard multivariate techniques we consider 
are based on the assumption that the underlying distribution for the observed 
multivariate data is multivariate Gaussian. It is well known that the univariate 
Gaussian distribution is bell shaped; however, the shape of the multivariate 
Gaussian normal is more complex and impossible to depict graphically for dimensions 
greater than three. The bivariate normal distribution can be described as having 
an elliptically mounded shape, and suitable cross-sections are, in fact, bell-shaped. 



Example 1 . (Based on Tatsuoka (p. 82, 1971).) Suppose in testing anti-anxiety 
agents, A and B, two hypothetical rating scales are employed: perceived stress 
and anxiety (where for both scales higher scores indicate higher perceived stress 
or anxiety). Ten patients are randomly divided into two equal groups, one group 
to receive A and the other B. At the end of six weeks of treatment, perceived 
stress scores and anxiety scores are obtained for each of the ten patients, with 
the data pairs as follows: 


Drug Group A Drug Group B 
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In this case, we have two samples of size 5 of bivariate (q = 2) observations. 
The multivariate means for Group A and Xg for Group B are 


X a = 


14.2 
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and respective sample variance-covariance matrixes and Sg are 
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Note that the sample correlations between perceived stress and anxiety for 
drug Group A and B are .87 and .96, respectively. 


MULTIVARIATE METHODS: UNIVARIATE EXTENSIONS 
One type of application of statistics often encountered is making inferences 
concerning the means of the variables of interest. The usual univariate methods 
dealing with this situation include among others Student's t, analysis of variance, 
analysis of covariance and regression. Corresponding to each of these univariate 
procedures is a multivariate extension. 


Student's t is most usually employed to test whether the means of two groups 
(or populations) are the same. For example, if one were comparing the effects of 
two drugs on a given set of laboratory parameter, an independent samples Student's 
t may be employed. The multivariate procedure corresponding to Student's t 
is Hotelling's T^. Suppose is a sample of size n from a q-variate 

multivariate Gaussian distribution and Ti»...,Y m is a sample size m from a 
q-variate multivariate Gaussian distribution. Hotelling's T^ can be used to test 
whether or not the q-variate population mean of the first sample is equal to the 
q-variate population mean of the second sample. The crucial consideration here is 
that the q means are collectively tested, one population against each other. 
Hotelling's T^ yields a single statistic which allows the simultaneous comparison 
of the q means from the first population with the corresponding q means from the 
second population. Consider studying, for example, the possible effects of two 
drugs on the change in the three liver laboratory parameters SGOT, SGPT and LDH. 

It would be appropriate to use a Hotelling's T^ to compare the values from one 
drug group with the other. This would jointly test possible drug effect on these 
three parameters taking into account the correlations among them. 


Example 2 . For the data given in Example 1, we wish to test whether or 
not individuals receiving drugs A and B have the same mean perceived stress scores 
and anxiety scores after six weeks of treatment. First note that if univariate 
Student's t test were conducted for each score separately, the resultant levels of 
significance for differences between drugs are p £ .55 for perceived stress, and 
p £ .012 for anxiety. The level of significance from Hotelling's T^ which jointly 
tests perceived stress and anxiety is p £ .002. Note that the multivariate level 
of significance in this example is smaller than either univariate level for this 
pair of highly correlated variables. 

Univariate analysis of variance is used to test for differences among the 
structures of numerous population means. The corresponding multivariate procedure 
is termed, appropriately, multivariate analysis of variance. While the mathematics 
of deriving multivariate analysis of variance tests are involved, the interpretation 
of the test statistics can be made reasonably readily by analogy with the corresponding 



univariate test statistics. For the chronic toxicity example in the Introduction, 
the multivariate version of one-way analysis of variance could be employed to test 
collectively whether or not there is any difference among the four treatments for 
the entire set of 52 weekly body weights. The one-way multivariate analysis of 
variance in this case would yield a single level of significance comparing all four 
treatments based on the year's weekly body weight data. (More typically in treating 
data of this type the weekly data would be averaged to obtain monthly body weight 
data and then comparisons made based on the monthly data.) If, in this example, 
there were two months' weekly body weight data for each rat prior to treatment, one 
could employ multivariate analysis of covariance to test for treatment differences 
after adjusting for each rat's pre- treatment body weight data. The interpretation 
of these results would be similar to univariate analysis of covariance. 

Univariate regression theory allows the study of the effect of several 
independent variables on one dependent variable. In an industrial chemical 
process, one could use regression analysis to study the effect of temperature, 
pressure and purity of raw materials upon yield. In multivariate regression 
theory, more than one dependent variable is allowed. For example, multivariate 
regression theory would permit the assessment of the effect of temperature, 
pressure and purity of raw materials upon yield and finished product purity, 
jointly. Any correlation between yield and finished product purity would be taken 
into account by the analysis. A multivariate regression equation for this chemical 
example would be of the following form: 


Yield 
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X Pressure 4- 
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MULTIVARIATE METHODS: STUDYING RELATIONSHIPS 

One of the essential reasons for a multivariate approach to statistical 
analysis is to be able to study and understand the relationships within a collection 
of variables. By the essence of multivariate analysis, there is no analogue of 
this concept within univariate statistics. Depending on the type of relationship 
being considered, one usually defines a suitable numerical measure that quantifies 
the strength of the relationship. The commonly employed measures are intuitive, 
and many of their mathematical and statistical properties are known and tabulated. 

A broad class of these methods, often called correlational methods, are intimately 
tied to the multivariate Gaussian assumption, while other classes of methods are 
much less dependent on the data having a Gaussian distribution. It is the former 
class of methods that are discussed in this section and a brief discussion of the 
latter is given later. 

In the case of two variables, the strength of the relationship between them 
depends on how well one variable predicts the other. The strongest possible 
relationship occurs when knowledge of one variable perfectly predicts the other 
and the weakest possible relationship is when knowledge of one variable provides 
no information concerning the other. For example, if one variable represents an 
object's length in inches and the other variable in centimeters, then these two 
variables are perfectly related (in fact, perfectly linearly related). On the 
other hand, it is clear that the annual California wine production is totally 
unrelated to the number of home runs hit in the World Series. 

The correlation coefficient, with a numerical range of -1 to +1, is the 
most commonly used measure of relationship when dealing with two variables. If 
its value is 1 (or -1), then there is a perfect positive (or negative) relationship. 
Moreover, this perfect relationship is a linear relationship. A value of zero 
indicates a complete lack of relationship. Intermediate values describe degrees of 
relationship so that a correlation of .83 indicates a stronger relationship between 
two variables than a correlation of .27. In Example 1, the sample correlations of 
.87 and .96 both indicate a high degree of relationship between perceived stress 
and anxiety, with drug Group B showing an apparently slightly stronger relationship. 
It is noted that a positive correlation coefficient indicates that essentially both 
variables are increasing or decreasing together, while a negative correlation 
indicates that if one variable is increasing then the other is decreasing. When 
comparing the strength of relationship between a positive and negative correlation, 
the absolute values should be used, i.e., ignore the minus sign on the negative 
correlation. 
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While correlation allows the measurement of relationship between two variables, 
it sometimes is desired to measure the relationship between two sets of variables. 

For example, in a 20 item questionnaire, the first 8 items may be associated with 
a Factor 1 and the remaining 12 items associated with a Factor 2. In studying the 
relationship between Factor 1 and Factor 2, one is examining, in essence, the 
relationship between the first 8 variables and the remaining 12 variables. The 
canonical correlation coefficient measures this type of relationship. Its properties 
are similar to those of the correlation coefficient, ranging from -1 to 1 and with 
a value of zero indicating no relationship. A value of .64 implies in this 
example that there is some weighted sum of the first 8 items that has a correlation 
of .64 with some weighted sum of the other 12 items, and that the highest possible 
correlation achievable in such a weighted sum fashion is .64. The actual weights 
that comprise the sum may be interpretable and the weighted sum variables are 
called the canonical variables. In the special case where the first set of 
variables has only one member, the canonical correlation coefficient is called the 
multiple correlation coefficient. 

On ocassion, one measures relationships between variables or set of variables 
when there is knowledge or conditions about an additionally related variable. 

Suppose that in addition to the questionnaire values for an individual, I.Q. is 
also available. One may wish to study the relationship between two questionnaire 
items under the condition that the individual have an I.Q. of 140. The partial 
correlation coefficient measures such conditional relationships. Much less 
commonly employed, but still useful is the conditional canonical correlation 
coefficient . 

MULTIVARIATE METHODS: STRUCTURE 

In this section, we consider some methods for understanding the underlying 
structure of the multivariate data. In a univariate situation, the data variable 
itself fully and completely describes the underlying data. For multivariate data, 
it's possible that the full set of variables are required to describe the basic 
underlying structure of the data. However, it's often possible to find a simpler 
set of variables than the full set that fully depicts the basic structure of the 
data. This simpler set of variables may be a sub-collection of the original set or 
may be certain combinations of the original variables. For this reason, the 
methods we consider are often called dimensionality reduction methods, i.e., they 
permit the studying of a set of q variables by examining a smaller set of variables. 
For the chronic toxicity example, the 52 weeks of data is a large set of variables 
with which to deal. It would be highly desirable to be able to find a much 
smaller set of variables that contains essentially the same information as the 
original variables. 

Structural multivariate methods can be viewed as falling into two general 
classes depending on whether or not comparisons across groups are important. In 
the. non-comparative case where one is interested in understanding the structure 
of a given set of data, there are the two wellknown techniques of principal 
components and factor analysis. Both methods seek to represent the structure or 
variability in a q dimensional set of variables in terms of a smaller number of 
variables each variable having zero correlation with another. Principal components 
achieves this by considering combinations of the original variables while factor 
analysis postulates as the reduced set of variables unknown factors which require 
interpretation . 

From a purely statistical point of view, factor analysis more formally 
takes into account the desired reduction in dimensionality but requires more 
interpretation, while principal components less formally handles the dimensionality 
reduction, but does not require as much interpretation. 

For a given set of data, both principal components and factor analysis 
depend only on the sample variance - covariance matrix from that data. While both 
methods should be employed for large sets of variables, e.g., 20 item questionnaire 
or 52 weeks of body weight data, we consider a simple example illustrating principal 
component analysis. 

Example 3 . For the data in Example 1, we wish to find for Group A the 
principal components corresponding to the perceived stress scores and anxiety 
score. The first principal component is the variable equal to .94X stress + .34 
X anxiety and the second principal component is the variable equal to .94 X 
anxiety - . 34 X stress. The variance of the first principal component is 77.38 
and of the second is 2.32, with the principal components being uncorrelated. A 
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standard interpretation of these results would be as follows. Because 77.38 is so 
much larger than 2.32, the full information for both variables stress and anxiety 
is "essentially carried" in the first principal component. This first principal 
component places more emphasis on perceived stress than anxiety, but is difficult 
to interpret further. 

When dealing with underlying structure when comparisons between groups are 
important, the techniques of discriminant analysis and canonical variables are 
appropriate. Canonical variable analysis is a dimensionality reduction technique 
while discriminant analysis is technically not such a technique. However, we 
include discriminant analysis in this section because of its close affinity to the 
other methods discussed. 

In the case of 2 groups (as discussed in an earlier section) , canonical 
correlation and canonical variables are a measure and variables, respectively, 
that relate the structure of one group with that of the other in such a way as to 
maximize certain correlations. As in principal components, there are multiple 
canonical variables (each uncorrelated with each other) for each group. The 
canonical variables for each group are weighted sums of the original variables for 
that group, with the correlation between the first canonical variables, i.e., the 
canonical correlation, being as large as possible. In essence, these two first 
canonical variables indicate the underlying variable in one group that is most 
related to an underlying variable in the other group. 

Suppose based on previously obtained data, sample multivariate means and 
variance - covariance matrices were available for each of several groups. 

Discriminant analysis allows the classification of a new data point into one of 
these several existing groups. For example, suppose based on a thorough medical 
examination, a group of school-age children was broken into four groups: normal, 
mildly hyperkinetic, moderately hyperkinetic, and severely hyperkinetic. A 20 item 
hyperkinesis questionnaire was administered to each group with means and variance - 
covariance matrices being computed. Now based on the questionnaire data from a new 
child, one desires to classify that child into the group which the child belongs. 
Discriminant or classification methods classify the child in such a way as to 
minimize a misclassif ication . Typically, one inputs to a discriminant procedure 
prior feelings about the relative sizes of the groups and also rough costs associated 
with a misclassif ication, e.g., classifying a normal child as hyperkinetic or vice 
versa . 

Cluster analysis and path analysis are two other structural methods worth 
mentioning. Cluster analysis is a set of techniques that allows one to take a 
multivariate data set and sub-divide the data into relatively homogenous groups. 

There are numerous clustering techniques with substantiative differences among 
them. Path analysis studies the relationships among observed variables and 
hypothesized underlying variables where the general form of the relationships 
(path diagram) must be specified in advance. 

MULTIVARIATE METHODS: MISCELLANEA 

Most of the commonly used methods previously discussed substantially rely on 
the fact that the data has an underlying Gaussian distribution. There is a virtual 
cornucopia of methods that assume other underlying distributions or just general 
forms of underlying distributions. It would be impossible to discuss more than a 
few such techniques here. 

In catagorical data, an entry in the multivariate array would only indicate 
into what category that entry falls. Suppose one has a three dimensional data 
array where the first entry is 0 or 1 if the patient improved or not, the second 
entry is 1, 2 or 3 according to whether the patient received drug A, B or C, and 
the third entry is 1, 2, 3 or 4 according to the staging of initial severity of 
that patient's disease. Log-linear categorical analysis permits one to study the 
individual or combined interrelationships of such data, relating, for example, 
improvement to treatment to disease severity. 

In dealing with measures of relationship for non-Gaussian data, there are 
innumerable "measures of association," or types of association each with different 
properties, pros and cons. For illustrative purposes, we list a few: rank 
correlation coefficient (useful for nonparametric or rank data), total positivity 
of order 2 (useful in reliability theory) and maximal correlation (a non-Gaussian 
analogue of canonical correlation) . 
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For handling multivariate data in a "life- testing” context, there is multi- 
variate reliability theory. There are, for example, several multivariate extensions 
of the important univariate exponential distribution. These multivariate exponential 
distributions permit modeling of complex systems where individual components have 
correlated times to failure, but components viewed by themselves have an exponential 
failure time. 


COMMENTS 

It should be stressed again that the methods given have been presented at an 
intuitive level. Heuristic arguments and "illustration by example" have been 
liberally employed. Very little emphasis was placed on formality and particularly 
on the necessary assumptions for each method. Obviously one should be aware of 
some of the more formal details in using any of these procedures. 

There are many very good texts for further reading in multivariate analysis 
ranging from elementary to highly mathematical. Included among the former category 
are - Harris [1975], Tatsuoka [1971] and Van de Geer [1971], and included among the 
latter are Anderson [1958], Dempster [1969], Press [1972] and Roy [1957]. Other 
references that combine both theory and application include Bishop, Fienberg and 
Holland [1975], Bock [1975], Gnanadesikan [1977], and Morrison [1967]. Each of 
these noted references would provide directions for further readings. 

There are at least three multi-purpose statistical program packages that 
include the basic multivariate statistical programs. They are BMDP - 1977, 

SAS - 1976 and SPSS - 1975. Clyde's [1969] MANOVA is purely a multivariate 
program. For references to other multivariate statistical programs, see the 
references, especially Press and Gnanadesikan. 

By its very nature, multivariate statistical analysis has a richness not 
found in univariate statistics. It has been our attempt in this overview paper 
to convey this richness by presenting some of the common, but highly useful 
multivariate techniques. 
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QUALITY ASSURANCE IN ENVIRONMENTAL REGULATIONS 


D. E. Splitstone 
U. S. Steel Research Laboratory 
Monroeville, Pennsylvania 15146 


INTRODUCTION 

A common complaint by practicing statisticians is that engineers 
and scientists feel perfectly capable of "doing statistics" while at 
the same time believing a statistician is totally incapable of under- 
standing the technical aspects of their problems. In discussing the 
role of the statistician in environmental law, I am fully aware that 
what I say may be viewed as the irrelevant ramblings of a scientist- 
meddler by many who formulate and interpret environmental policy. Yet, 
I am also aware that these same persons take as part of their license 
the interpretation of complex technical evidence, often without the 
benefit of full and complete consultation with unbiased experts. This 
situation has always been mystifying to me, as is the question of why 
environmental regulations do not, in general, reflect in their language 
even basic statistical principles. The answer is complex, revolving 
around a communications problem between those engaged in lawmaking and 
law enforcement and those who provide the scientific basis for policy 
formulation . 

The space alloted allows only a cursory glance at the complex area 
of environmental regulation. If this paper generates sufficient 
interest to spur some to investigate further and take some action, then 
its objective is accomplished. 


DISCUSSION 

The regulatory process in general, and the environmental regulatory 
process in particular, is necessarily a judgmental process. Scientific 
evidence can be and has been used to help demonstrate a need for regula- 
tion. It can be and has been used to help define what is to be regulated 
and the extent of the regulation in terms of standards or criteria for 
performance. However, we all realize that scientific data are not 
perfectly definitive in general and are subject to interpretation.. The 
statement that compound XYZ is a carcinogin begs the questions, to what 
population, what are the risks to this population, and how do these 
risks stack up against potential benefits? If it is decided to regulate 
compound XYZ, what criteria should be used for the regulation? What 
levels of XYZ can be tolerated in the environment? Are the social and 
economic costs of control justified by the risks? Judgments must be 
made in answering these questions. These judgments often are not 
purely scientific, but are political in nature. 

It is obvious that the scientist has a place in providing input to 
this judgmental process through reporting the results of his work, 
hopefully with appropriate caveats. Most of us feel that we have 
discharged our professional responsibility when we have presented the 
results of our work to the environmental-policy formulator. We are 
then all too eager to back off and let the policy makers and regulation 
formulators go about their business. These decision makers, in turn, 
are perhaps very pleased to have the scientist, with his cautions 
concerning the interpretation of his work, out of their hair. 
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As this paper concerns statistics and environmental law, perhaps 
"statistics" should be defined. A popular dictionary 1 ^ contains three 
definitions : 

1. "The mathematics of the collection, organization, and inter- 
pretation of numerical data; especially, the analysis of 
population characteristics by inference from sampling..." 

2. "A collection of numerical data..." 

3. "Political science dealing with state affairs..." 

It is this third definition which is the major topic of discussion. 

The first definition, however, should also be considered as it clearly 
states that statistics is a branch of mathematics. Indeed the same 
dictionary 1 ' 1 defines "statistician" as "A mathematician specializing in 
statistics . " 

The use of language in mathematics is, or should be, precise. It 
would be reasonable, then, to expect at least as much care in the use 
of the language of statistics as in the use of mathematical language. 

Such care would be expected particularly of the law, which often focuses 
on the interpretation of language. Some would even declare that greater 
care was essential in the use of statistical terminology because 
statistical terminology must often deal with uncertainty and inductive 
inference . 

Consider, now, the language used in the specification of -the 
limitations for the average of daily water effluent contaminant values 
for any period of thirty (30) consecutive days. First, consider the 
limitations for the nitrogen fertilizer industry. It can be inferred 
from comments (14) and (22) on page 12834 of the April 8, 1974, Federal 
Register that these limitations represent, in fact, the average discharge 
attained by exemplary plants within each category or subcategory. 

Quoting the last sentence of comment (14) 

"Limitations for ammonium nitrate, therefore, have been revised to 
the average of the best levels currently being achieved by plants 
not using ion exchange." 

Quoting from comment (22) 

"Congress, in the Agency's view, intended that the best practicable 
control technology currently available be determined by reference 
to the average of the best existing performances by plants of 
various sizes, ages, and unit processes within each industrial 
category. This average is to be based not on a broad range of 
plants but on the performance levels attained by exemplary plants 
within each category or subcategory." 

A similar statement for the iron and steel industry can be found 
in Section iii of the proposed rules given on page 6492 of the February 
19, 1974, Federal Register. 

"The unit effluent load limitations were developed by determining 
the minimum unit effluent volumes that could be achieved by the 
application of good water use practices and by a determination of 
residual pollutant concentrations that remain after application of 
the appropriate treatment technologies. The product of these is 
the unit effluent load limitations proposed. The limitations thus 
developed represent values not to be exceeded by any thirty (30) 
consecutive days average..." 



It is a fundamental concept that average values being calculated 
from measurements having a distribution can be expected to have a 
distribution also. The conceptual population of averages has as the 
true average of its distribution the true average of the distribution 
of the original measurements. The variability of the averages is 
reduced from the variability of the original measurements in proportion 
to the square root of the number of measurements averaged. 

In each case it is assumed that if an operator installs the best 
practicable control technology currently available and operates it 
properly, then the average of daily discharge measurements for every 
period of thirty (30) consecutive days will be the same. Ignored in 
the statement of these regulations is the fact that the average of the 
daily discharge measurements for a period of thirty (30) consecutive 
days is an estimate of the true average performance of the facility 
and, therefore, exhibits some variability. The operator is then left 
with a 50 percent risk of violation. 

Dr. J. Stuart Hunter has, during his work on the environmental 
measurement problem, uncovered many other examples. Dr. Hunter stated 
the following examples during his presentation to the American Statis- 
tical Association. ) 

"Consider the description of the EPA reference methods for the 
criteria air pollutants that appears in the Federal Register, Title 40, 
Appendix A 4. We read under the caption 'Precision, Accuracy, and 
Stability' for S0 2 (36FR8188) , the following, 'Relative standard devia- 
tion at the 95 percent confidence level is 4.6 percent for the analytical 
procedure using standard samples...' We can interpret 'relative standard 
deviation' to mean the coefficient of variation, but the phrase 'at the 
95 percent confidence level' is simply undecipherable: Too many 

interpretations are possible. In an attempt to unfold the meaning of 
this statement, the literature referenced in the Federal Register was 
checked only to discover that the estimate of the 'relative standard 
deviation' had been based on three observations (!) . The statistical 
statement in the Register is not only unclear, it is based upon sketchy 
information. Continuing in the same issue of the Register , under the 
caption, 'Precision and Accuracy,' we find the following: 

On Suspended Particulates 

'4.1 Based upon collaborative testing, the relative standard 
deviation (coefficient of variation) for single analyst 
variations (repeatability of the method) is 3.0 percent.' 

On CO: 

'4.1 Precision is +0.5 percent full scale with 0 to 58 mg/m 3 
range . ' 

and again later under definitions: 

'Precision: The degree of agreement between repeated measure- 

ments of the same concentration, expressed as the average 
deviation of the single result from the mean. ' 

On Photochemical Oxidants: 

M.l The average deviation from the mean of repeated single mea- 
surements does not exceed 5 percent of the mean of the 
measurement . ' 



On Hydrocarbons: 


'4.1 Precision determined with calibration gases is ±0.5 percent 
of full, scale in the higher, atmospheric analysis ranges.' 

Thus, for each criteria air pollutant we have unique measures of 
precision . 

Or consider the following nonsequitor from the Federal Register, 

10 September 1976 under fuel economy testing; calculation and exhaust 
emissions test procedures for 1977-1979 model year automobiles. Interim 
Final Rule Making and Proposed Rule Making: 

'Unlike statistical approaches to sampling, representative sampling 
plans rely on the testing of enough cars within each base level so 
that the fuel economy of production vehicles within each base 
level is adequately represented by the vehicle tested. ' 

As Professor Deming has stated, the paragraph begins by scorning 
statistical sampling, and then proceeds to describe a stratified 
sampling plan. It would appear that no knowledge of statistics is 
required before the language of the art is employed." 

Thus the language, which should result from definition 1, does not 
appear in resulting official regulations and procedural descriptions. 

Why is this the case? Is it because the practioners of statistics have 
themselves ignored the implications of definition 3? Or, is it because 
the lawmakers do not think it is necessary to heed the pronouncements 
of the statistician because they are perceived as irrelevant to the 
lawmaking process? Perhaps the answer is a combination of the two. 

As lawyers want to look for precedents, let us briefly consider 
whether there are precedents for a close association between science 
and the law. When I attended the 1976 annual meeting of the Natural 
Resources Law Section of the American Bar Association as a technical 
observer, along with an EPA statistician, we heard topics concerning 
environmental law discussed with great interest and gusto. However, 
nowhere in the discussions was the importance of the quality or quantity 
of technical input considered. Is it then that technical and scientific 
input has no place? 

Lee Loevinger in his concise paper, " Jurimetrics : Science in 
Law, " indicates that the use of science in seeking answers to social 
and legal problems has long been urged by eminent scientists and lawyers 
alike. Karl Pearson, writing in 1892, expressed the spirit of his 
age as follows: 

"The classification of facts and the formation of absolute judgments 
upon the basis of this classification— judgments independent of the 
idiosyncracies of the individual mind — essentially sum up the aim and 
method of modern science. The scientific man has above all things to 
strive at self-elimination in his judgments, to provide an argument 
which is as true for each individual mind as for his own. The classifi- 
cation of facts, the recognition of their sequence and relative signifi- 
cance is the function of science, and the habit of forming a judgment 
upon these facts unbiased by personal feeling is characteristic of what 
may be termed the scientific frame of mind. The scientific method of 
examining facts is not peculiar to one class of phenomena and to one 
class of workers; it is applicable to social as well as to physical 
problems, and we must carefully guard ourselves against supposing that 
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the scientific frame of mind is a peculiarity of the professional 
scientist . " 

The same views were expressed by the great lawyer and jurist, 

Oliver Wendell Holmes, 5 ) in 1895 when he said 

"An ideal system of law should draw its postulates and legislative 
justifications from science. As it is now, we rely upon tradition, or 
vague sentiment, or the fact that we never thought of any other way of 
doing things, as our only warrant for rules which we enforce with as 
much confidence as if they embodied revealed wisdom." 

Holmes^' 7 ^ reiterated this theme more than once, and he spoke for 
a school and a generation of legal realists when he declared 

"For the rational study of the law the black-letter man may be the 
man of the present, but the man of the future is the man of statistics 
and the master of economics." 

"...The growth of education is an increase in the knowledge of 
measure. To use words familiar to logic and to science, it is a sub- 
stitution of quantitative for qualitative judgments ... In the law we 
only occasionally can reach an absolutely final and quantitative deter- 
mination, because the worth of the competing social ends which respec- 
tively solicit a judgment for the plaintiff or the defendant cannot be 
reduced to number and accurately fixed. The worth, that is, the 
intensity of the competing desires, varies with the varying ideal of 
the time, and, if the desires were constant, we could not get beyond a 
relative decision that one was greater and one was less. But it is of 
the essence of improvement that we should be as accurate as we can. 

"...I have had in mind an ultimate dependence upon science because 
it is finally for science to determine, so far as it can, the relative 
worth of our different social ends, and, as I have tried to hint, it is 
our estimate of the proportion between these, now often blind and 
unconscious, that leads us to insist upon and to enlarge the sphere of 
one principle and to allow another gradually to dwindle into atrophy. 
Very likely it may be that with all the help that statistics and every 
modern appliance can bring us, there never will be a commonwealth in 
which science is everywhere supreme. But it is an ideal, and without 
ideals what is life worth?" 

This view that science and law have a close relationship and that 
science must contribute to the formulation of law continues to be 
expressed. However, as C. P. Snow a admits, "it is not always easy to 
know which scientist to trust." This points to a problem perhaps more 
important than that of language. A scientist who is a zealot is no 
different from anyone else who is a zealot. Often, for each "scientist" 
expressing a point of view, another can be found who will express with 
equal zeal the opposite point of view, with both opinions based on the 
same data. Perhaps worse than the zealot scientist are those who for 
reasons of increasing their list of publications, continue to turn out 
reports of questionable value under the guise of scientific papers. 

As scientists concerned about quality environmental regulation, 
what can we do to have the maximum impact on these regulations? First, 
we must gain a better understanding of the political processes through 
which society makes its decisions and take a much greater interest in 
these processes. We cannot stand back after making our scientific 
pronouncements and let the politician take over. We must assume our 
responsibility as practioners of statistics which is inherent in the 
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third definition. That is the responsibility for insisting on quality, 
responsibility for curbing misuse, responsibility for adequate descrip- 
tion and for stressing limitations. We must insist that these respons- 
ibilities be met by all practioners as well. 

Second, we must work toward better communication and appreciation 
of statistical concepts with the lawmakers. This will help to overcome 
the language barrier. The result will be better standards and more 
reasonable regulations. 

And lastly, we must, through our professional associations, become 
more active in expressing our opinions. By speaking out as an associa- 
tion of professionals the real or imagined advocacy bias can be avoided. 
Professional associations must also make their views known to the 
lawmakers in the course of public hearings, or perhaps through lobbying. 
The American Society for Quality Control is currently doing this in the 
area of consumer product standards. This activity must be expanded to 
other areas as well. 

The need for scientific knowledge in policy-making institutions is 
increasing at an increasing rate. Scientific data are clearly not only 
relevant but crucial to the consideration of many of our most pressing 
contemporary social problems. 

The Government has been rapidly increasing its scientific manpower, 
particularly in the environmental area. Before we take comfort in 
this, note the following caution from Loevinger ' s article. ^ 

"When science becomes a part of government, it is the science that 
is corrupted, not the government that is ennobled. As a speaker at the 
1971 AAAS annual meeting pointed out; 

'To the extent that the scientific enterprise becomes enmeshed in 
this (governmental) network, its critical distance is diminished. 
Independent, critical work is dampened and impeded where there 
exist obligations or close connections to those in power. What 
this means is that one cannot serve two masters — at least, not at 
the same time. A commitment to theorizing, for example, precludes 
involvement in the practical world of policy-making.'" 

In a democracy, the most appropriate and promising method for 
incorporating scientific knowledge into public law is public dialogue. 

If we as professionals do not take the opportunity to provide quality 
input to the policy-making system, then the void will be filled by 
others who are perhaps less concerned with such quality. 


It is understood that the material in this paper is intended for general 
information only and should not be used in relation to any specific 
application without independent examination and verification of its 
applicability and suitability by professionally qualified personnel. 
Those making use thereof or relying thereon assume all risk and 
.iability arising from such use or reliance. 
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QUALITY COSTS - WHAT DOES MANAGEMENT EXPECT? 
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INTRODUCTION 

The measurement of quality cost has proven its importance in the 
quality control field (1) . However, my experience indicates that only 
about 40% of the American companies have a system that goes beyond 
measuring scrap and rework dollars. Furthermore, some companies do 
not even dollarize scrap and rework. Thus, for many companies quality 
costs still represent an opportunity. 

One of the factors influencing the introduction of a quality cost 
system in a company is the attitude of top management. This paper will 
look at quality costs from the viewpoint of upper management levels — 
plant manager to president. This will be done by describing ten actual 
situations ("scenarios") and offering some conclusions for each. 

SCENARIO #1 - WHAT ARE QUALITY COSTS? 

A quality control manager decides to measure quality costs. He 
works with the accounting department and proposes that quality costs be 
compiled for four basic categories: prevention, appraisal, internal 
failure and external failures. The accounting people declare that they 
certainly want to cooperate with him but "the books aren't kept that 
way." They also point out that scrap and rework dollars have long been 
known and are rather stable at about 2% of direct labor and direct 
material costs. As the 2% is included in the standard cost system, the 
accounting people feel that scrap and rework costs are covered and there 
is no need for further measurement. 

Finally, the data is collected and presented to the plant manager. 
His initial reaction is "these are the costs of the quality control 
department and they certainly are too high." This misunderstanding 
horrifies the QC manager. Next, the manufacturing manager refutes the 
data because he claims that part of the "scrap" is unique to the type of 
manufacturing process and clearly unavoidable. Meanwhile, the account- 
ing people continue to emphasize the inclusion of scrap and rework 
dollars in standard costs. The plant manager is confused. 

Conclusions . Top management expects that any presentation of 
quality costs "will clearly identify the meaning of the costs. Further, 
they expect that any arguments about including certain elements of costs 
such as unavoidable manufacturing scrap will be settled before present- 
ing the final report. The misunderstanding that quality costs are the 
same as the costs of the quality control department is an offshoot of 
the idea that the quality program in a company is the same as the work 
of the quality control department. We know that this is not the case 
but, like it or not, many people believe it. It's probably too late to 
change the term quality cost but you may want to consider using another 
term in your own company. 

We need to say what quality cost is and what quality cost is not. 
Quality cost is a collection of costs throughout the entire company 
associated with the prevention, generation, measurement, and correction 
of quality problems. Quality costs are not equal to the costs of the 
quality control department just as federal expenses are not equal to 
the expenses of the Treasury Department. Quality costs go beyond the 
standard amount for scrap and rework included in standard costs. How- 
ever, the standard cost system assumes that such costs are inevitable — 
the QC system views these costs as cost reduction opportunities. 
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Avoidable and unavoidable costs in connection with quality should 
be clearly labeled. In addition, careful explanation is needed for any 
"hidden costs." For example, suppose a packaged product must meet a 
minimum weight requirement. If the process variability is wide then the 
process average must be set sufficiently above the minimum weight to 
have a high probability of meeting the minimum on individual packages. 
Although the product meets the specification, the high process average 
represents a giveaway which is a hidden cost of quality. 

SCENARIO #2 - PAR 

A QC manager presents his first quality cost study and shows that 
quality costs are equal to 11% of sales dollars. There is general 
agreement by upper management that the figure is high but top management 
asks "what should it be?" The QC manager replies that it is difficult 
to have a specific numerical answer but the industry average is about 
6%--he read this in a magazine article. Now the arguments begin. Some- 
one points out that the 6% probably varies by type of industry. Another 
argument is that the amount of quality cost depends on the quality 
philosophy of the company — is the company aiming to be the top quality 
leader or just average with competition? Another question is how dif- 
ferent companies define quality costs — is overhead included, is unavoid- 
able manufacturing scrap included, are periodic losses of new contracts 
due to poor quality image included? These questions are a mixture of 
wind, snow, ice, and trouble. 

Conclusions . Management not only wants to know the total quality 
cost but needs to know how to interpret the total. Comparison to other 
companies or industry in general can be dangerous for the reasons cited. 

A better tact is this: select a few major projects to improve, 
dollarize the size of the quality costs on these projects, set a cost 
reduction goal, estimate the investment required and calculate a measure 
of effectiveness such as return on investment or payback period. Of 
course, the complete participation of responsible departments in all 
this is a must. 

Finally, trends may be more important than the absolute value of 
quality cost. They can serve as an early warning device and help to 
set specific goals for the future. 

SCENARIO #3 - WHAT SHOULD I DO? 

The QC manager of a consumer product presents a quality cost report 
including the four basic categories. The executive vice-president re- 
views it and agrees that the overall costs are high and that the cate- 
gories make good reading but then nothing happens. Finally, he asks 
"what should I do with this information?" 

In another plant making a product that is a rental product, the 
quality cost report to management is extremely detailed. Many ratios 
highly useful to QC professionals are presented and the emphasis is both 
on investment costs and losses. Again, the quality cost report simply 
is not getting enough action from upper management. 

In two other cases, upper management express a desire to relate 
quality costs to objectives for functional managers, e.g., through a 
tie-in to bonus plans or a management by objectives program. 

Conclusions . Management expects to be shown how to interpret 
a quality cost report and the benefits and costs of alternative actions. 
QC measures quality. This scorekeeping is necessary but management 
expects somebody to show them how to win the pennant. Specific improve- 
ment projects with numerical objectives must be defined as part of the 
proposed action to be taken. This is in contrast to just dramatizing 
the high total and asking for general support across the board on 
quality . 

The improvement projects can be on products or on tasks within a 
quality program. For example, objectives for next year on specific pro- 
ducts might be "quality costs for the company shall be reduced by 20%" 
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or the "average leak rate for product x shall be reduced to 4%." 
Examples of objectives for tasks rather than products might be "quality 
costs shall be determined for at least one product" or "numerical reli- 
ability and maintainability objectives shall be defined for at least 
one product." 


SCENARIO #4 - THE TOTAL 

A QC manager calculates quality costs as a total of $1.5 million 
for a plant with about 1,000 production workers. The cost is primarily 
scrap, rework, and warranty charges. He presents this information to 
justify his request for an increased budget next year. 

The plant manager reacts by agreeing that it is useful to collect 
all of the costs concerning quality across all departments and show it 
in one total. He agrees that the total is high but says that their 
product is relatively expensive and perhaps the absolute figure is not 
completely meaningful. After reviewing the literature, the QC manager 
restates the quality costs as 9.2% of the sales income and shows a 
breakdown of the total into the usual four basic quality cost cate- 
gories. There is much argument about the usefulness of all this for 
the top management people. (The president later told me that he did 
not recall ever hearing about the four categories — zero impact.) 

Conclusions . The total quality cost figure is often dramatically 
high. However, presenting this figure by itself to top management 
people generally is not sufficient. The figure must be related to 
sales or some other base and some breakdown of the figure should be 
made. The problem is that the nature of the breakdown often depends on 
the industry and on the specific management person involved (see 
Scenarios 5, 6, and 7). Useful as the prevention, appraisal and fail- 
ure categories are they may not be the right categories for your case. 

It may help to distinguish between fixed and variable quality 
costs. Other possible breakdowns are by product, vendor, department 
defect type or defect cause depending on the level of detail desired. 
Pareto scores again. 

Finally, quality cost information may need to be presented as part 
of a larger "instrument panel" which shows quality costs plus other 
quality indicators that can help management understand the significance 
of the dollar figures. The quality indicators used for top management 
of an equipment manufacturer are shown in Table 1: 


TABLE I. Example of Quality Indicators 
Internal quality indicators 

1 . Machine shop scrap and rework hours 

2. Assembly scrap and rework hours 

3. % of finished product which is incomplete at end of line 

4. Dollars of purchased material rejected 

External quality indicators 

1. Defects found on delivery to dealers 

2. Failure incidents per unit time 

3. User rating of product based on opinion poll 

4 . Dollars of warranty expense 


Each of these indicators is related to an appropriate base. Note 
that only 2 of the 8 indicators are in dollars. The primary reason is 
that a large share of the quality costs are in labor hours and this 
governs the emphasis on quality improvement. 

SCENARIO #5 - UNITS, UNITS, UNITS 
A group of QC managers are comparing notes on their experiences 
with quality cost. Bill comes from a chemical company and mentions how 
his division president calculated the cost of spoilage as 13d per share 
of stock outstanding. This was instrumental in convincing the presi- 
dent to direct corporate QC to set up a quality cost system throughout 
the division. 
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Tom comes from a mechanically oriented company where the quality 
cost for all divisions is $76 million per year. He comments about the 
value of converting this dollar figure into equivalent physical terms. 
For his case, the $76 million converts to 5,900 extra personnel, 1.1 
million square feet of space, and $6 million of extra inventory being 
carried. These figures collectively are equivalent to the entire yearly 
production of one plant of his company. This physical analogy had great 
impact on his management. 

Ed works for a food processing plant where rework alone is three- 
quarters of a million dollars a year. This figure was instrumental in 
convincing the plant manager to set up the training program for foremen 
in process control methods. In this case the effect on delivery prom- 
ises to customers was also an important factor in dramatizing the im- 
portance of the rework problem. 

Conclusions . Management expects us to present quality costs in a 
language suited to their needs. Often that language is dollars but not 
always. Sometimes productivity and delivery schedules or other matters 
are equally or more important. The need is to discover what language 
fits the territory. 

SCENARIO #6 - BASE, BASE, BASE 

The QC manager of a company that designs and manufactures mechani- 
cal assemblies prepares a quality cost study and explains the results 
to engineering, manufacturing, purchasing, and other major function 
areas . 

Everyone is impressed that the total quality cost is high but there 
is considerable difference of opinion on how to interpret the figure. A 
further discussion breaks down the total by plants but now the arguments 
really become heated. The different plants make different products and 
the only agreement reached is that the absolute quality cost dollar 
figure is not sufficient to make comparison between plants. 

A number of different bases are proposed such as expressing quality 
cost as percent of sales, quality cost as percent of factory costs, and 
so forth and the discussion continues into the night. 

Conclusions . Management expects the QC manager (in cooperation 
with others) to ' determine the most useful way to present quality cost 
figures. The literature emphasizes quality cost as percent of sales. 
However, other bases include direct labor hours, direct labor dollars, 
standard manufacturing cost dollars, value added dollars, burden, and 
product units. 

It is difficult to logically determine beforehand which base or 
bases to use. In my experience, several bases should be selected and 
tried for a period of time and then a judgment made on which bases are 
best . 


Figure 1 shows total quality costs expressed in absolute dollars 
and two different bases. Note how the conclusions vary with the base 
used. The data for this figure was taken from Reference 2 which in- 
cludes a further discussion of bases. 

SCENARIO #7 - MY PLANT IS DIFFERENT 
In my experience, the reaction of plant managers and company presi- 
dents concerning quality cost is quite varied. The president of an 
electrical component manufacturer is oriented to meeting the needs of 
the appliance industry and focuses on quality and sales income rather 
than quality and cost reduction. On the other hand, the president of a 
pharmaceutical company wants to control the cost of quality but is more 
concerned about the seriousness of a major recall on any of his products. 
The division manager of an appliance firm is more concerned about the 
customer getting value than he is about reducing quality costs. The 
president of a small firm producing lawn equipment echos the same belief. 
The plant manager of a chemical company explains that a directive from 
corporate headquarters set the stage for him: reduce the cost of 



nonconforming material. A vice-president of a high technology company 
is cost/benefit oriented. In his words: "I don't mind spending money 
for quality but I sure wish I could be more certain of my investment 
rather than spend it in a defensive manner." 

Conclusions . Management expects us to present quality costs in a 
way that is unique for their situation rather than emphasizing the con- 
ventional methodology of quality costs. If the orientation is toward 
reducing the cost of nonconformance, the conventional methodology will 
meet the test. Other priorities require a different tact. 

The priorities of the executive depend on the industry but also the 
scope of the responsibility. For example, a plant manager responsible 
only for manufacturing may stress conformance to specifications. A 
division or company president with broader responsibilities must weigh 
the cost of nonconformance with design, sales and other matters influ- 
encing profit. 

SCENARIO #8 - VALUE 

A discussion with a division manager for a major product line in 
the appliance industry makes it clear that his road to increased profits 
is to increase sales rather than just reduce quality costs. He feels 
that product value is the key to increased sales. For example, his 
company can provide a digital readout of temperature in the main com- 
partment of a refrigerator for about $40 additional in the price. How- 
ever, a study concludes this would be poor value from the customer's 
viewpoint and therefore the idea is dropped. 

The same manager also makes a distinction between the real and per- 
ceived performance of the product. The real performance of the product 
is what the product can actually do. The perceived performance is the 
performance as the customer sees it. For example, refrigerators are 
designed to have a certain temperature in the refrigeration compartment. 
If hostile environments are encountered (high humidity, high usage due 


to children) then excessive temperatures occur. It is possible to de- 
sign the refrigerator to handle these temperatures but it would not be 
economical — too few customers are involved. Now he relates this to 
quality cost. If such a refrigerator is marketed then the cost of serv- 
ice calls during warranty for the customers with extreme temperatures 
is viewed as a quality cost that is planned. 

This division manager reacts to questions about quality cost by 
directly orienting to value provided to the customer. This is not the 
intent of quality cost systems but it is his orientation. 

Conclusions . The quality cost concept as we QC professionals view 
it is too narrow for some management people. The management level that 
is responsible for manufacturing and design and sales expects customer 
talk and not just prevention, appraisal and failure costs. 

Table 2 shows an example of a method for comparing alternative de- 
signs of a product taking into account both performance and cost (i.e., 
value) . For each performance factor a fraction is calculated. The num- 
erator is a product of a weighting constant for the factor and the score 
achieved by the specific design for that factor. The denominator is the 
unit cost of achieving the factor by the specific design. Design B shows 


TABLE II. Value Comparison of Food Waste Disposer 


Performance 

Factor Design B Design G 


Grinding time 

9/0.87 

= 

10.3 

6/2.40 

= 

2.8 

Fineness of grind 

4/7.82 

= 

0.5 

4/11.88 

= 

0.3 

Frequency of jamming 

9/1.98 

= 

4.6 

9/2.46 

= 

3.7 

Noise 

4/0.45 

= 

8.9 

4/0.52 

= 

7.7 

Self-cleaning 

2/0.49 

= 

4.1 

4/0.58 

= 

6.9 

Elec, safety 

16/0.52 

= 

30.8 

16/0.43 

= 

37.2 

Particle protection 

6/0.30 

= 

20.0 

2/0.37 

= 

5.4 

Ease of servicing 

4/0.52 

= 

7.7 

6/0.98 

= 

6.1 

Cutter life 

9/0 . 83 

= 

10.8 

9/1.32 

= 

6 . 8 

Ease of installation 

9/0.33 

= 

27.3 

9/0.70 

= 

11.8 

Total 

72/14.11 

= 

5.1 

69/21.44 

= 

3.2 


more value than Design G. Such an analysis could compare alternative 
internal designs or an internal design with competition. 

Conventional quality costs measure costs incurred by the manufac- 
turer. This is separate from costs incurred by the user of the product 
(see Reference 3) which in turn is related to income. Although the qual- 
ity cost concept has proven its use as a cost reduction measure, manage- 
ment may increasingly look to the QC manager to relate quality and sales 
income. For example, in both the automotive and appliance industries, 
the share of the market given to suppliers for certain companies direct- 
ly depends on a quantitative quality rating which is computed periodi- 
cally. Suppliers whose quality rating is higher than competition are 
given a higher share of the market for the next purchasing period. 

One chemical company reported that there may be more potential for 
profit improvement (through increased market share) by improving the 
products supplied to the customer as contrasted to reducing scrap and 
rework costs. 


SCENARIO #9 - LET’S NOT GET BURNED AGAIN 
A manufacturer of major mechanical assemblies looks back on several 
new product developments. Classically each new product is carefully 
tested and evaluated before marketing (wouldn't it be nice if this 
really happened?) . However, on some new products the warranty costs were 
not only high but the company image suffered by product weaknesses that 
were not uncovered during the evaluation. This clearly hurt sales. In 
addition, the company has not been sufficiently prepared to handle the 
problems of servicing and provisioning of spare parts. Again, the image 
suffered — and this clearly hurt sales. 




Management said "let's not get burned again." Some new reliability 
efforts were instituted including the prediction of warranty costs as 
early as possible upon the introduction of a new product. 

Ironically, management now notes two payoffs: potential field prob- 
lems are anticipated and the prevention efforts have resulted in cost 
reduction. The original emphasis was to "get the customer off our back" 
but (pleasant surprise) this has been accomplished and a cost reduction 
also achieved. 

Conclusions . Particularly for companies making a diversity of new 
products, upper management expects the quality function to provide early 
warning devices on warranty costs and other quality related matters. 
These early warning devices must provide sufficient reaction time to 
prevent major field quality problems from occurring and to do this early 
enough to eliminate the need for compromising either costs or delivery 
schedules. Weibull Probability Paper for warranty cost analysis can be 
helpful . 


SCENARIO #10 - THE CORPORATE ROLE 

The president of a major division of a chemical company decides on 
the need for a quality cost system throughout his division and instructs 
the division QC people to proceed. The division QC people develop the 
system and are currently running a pilot run of the system in one plant. 
So far, the results look encouraging — the necessary data can be collect- 
ed in a reasonable time. A meeting on quality costs is held at another 
plant of the division and that discussion raises all kinds of questions. 
For example, the meaning of manufacturing scrap, avoidable vs unavoid- 
able costs, who is responsible for publishing the data, exactly what 
format will the data be in, what use will be made of the data by plants, 
and more . 

Conclusions . Management often looks to corporate QC people to 
develop and institute an overall quality cost system for the company. 
However, management knows that reductions in quality costs come from 
plant cafeterias rather than corporate dining rooms. Thus, the corporate 
people must work closely with the individual plants to develop the qual- 
ity cost system. Before procedures are issued, sufficient time should be 
allotted for extensive discussions with the plants to make sure that the 
plants can provide the data, and that the resulting reports will be use- 
ful both for comparing the different plants and for conducting the qual- 
ity improvement programs. Along with this should be the realization 
that the issuance of the reports alone will not secure a lasting reduc- 
tion in quality costs. Large quality costs should be used to justify 
and provide to the plants the additional manpower needed to determine 
the causes of the quality problems and institute corrective action. 

SUMMARY 

The determination of quality costs is still unknown in many com- 
panies. This presents an opportunity for the QC manager. However, the 
priorities of top management are quite varied even though the profit 
objective is common. The challenge for the QC manager is this: view 
quality as a business problem and adapt conventional quality cost 
methodology to meet overall company needs. 
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CAD/CAM - QUALITY'S MOST SERIOUS CHALLENGE 


B. J. Chumbley 
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INTRODUCTION 

Within the past two decades, the degree of advancement in the 
machined parts industry has been only one step short of miraculous. 

Today, with the aid of computer technology and other related technical 
advances, an idea can be translated from the mind of the designer to a 
finished machined part without one single piece of paperwork being 
created. Furthermore, the design and production phases are being accomp- 
lished at a fraction of the time previously required by conventional 
means. Mathematical calculations and machining cycles are performed by 
computers, thus reducing the chance for human error. 

One cloud hangs over this otherwise bright picture. This is the 
heretofore inability of the quality assurance discipline to maintain 
pace with the rapidly accelerating design and machine tool advancements. 
Traditional inspection methods will result in either an excessive amount 
of machine idle time for verification activities, or an abnormally large 
amount of scrap and rework when errors are not detected until final in- 
spection is performed. Quality Assurance activities must, therefore, be 
designed to take advantage of these technological advances, yet provide 
for the earliest discrepancy definition and correction activities 
possible, based upon economic considerations. 

The purpose of this paper is twofold: First, it will acquaint the 
reader with the development of the numerical control (N/C) discipline, 
beginning with the early attempts, progressing through the various stages 
of development and ending with projections for future evolutions. 

Secondly, the need for a revision in quality assurance discipline 
to control this new science will be emphasized. Since the concept of 
verification after machining is no longer economically feasible, verifi- 
cation activities must be advanced into the software portion of the pro- 
gram. Basic guidelines with viable alternatives will be discussed, with 
the final solutions left up to the discretion of quality management. 

THE BEGINNING OF NUMERICAL CONTROL 

William Sellers, President of William Sellers & Company of Phila- 
dephia, is credited with developing the first successful numerical con- 
trolled metalworking machine. Before his death in 1905, Sellers had 
outfitted a large multiple punching machine with a punched paper tape 
control unit. This machine was used to produce prefabricated bridge and 
other large structural forms at the Midvale Steel Company. Work con- 
tinued on the development of this concept by the Sellers Company and in 
1906, a large, pneumatically controlled 36 tool punching machine was 
placed on the market. With a capacity to handle steel plate 10 ft. wide 
and up to 100 ft. long, the machine would automatically punch rivet holes 
and other openings at predetermined spacings after receiving commands 
from its pneumatically operated punched card reader. 

Partially due to its size and mainly due to industry's preference 
for cam-controlled machinery as a means for automation, the Sellers 
automatic punch press was never a strong commercial success and was 
subsequently abandoned. 

In 1921, the second attempt to introduce a numerical controlled 
machine was made by Emanuel Scheyer. By adaptation of a control unit 
©1978 American Society for Quality Control 
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principle perfected by Joseph Marie Jacquard. Scheyer designed the first 
numerical controlled lathe. This rudimentary machine consisted of a 
bewildering accumulation of "gears, clutches, levers, and pneumatic 
cylinders to operate the cross slide and carriage as well as their feeds, 
traverse rates, and the spindle speed". Recognizing that this new con- 
cept could not actively compete with the more popular cam-operated 
machines, Scheyer designed his lathe for "handling work which is larger 
or more complicated than can be handled by cam-controlled machines." 

Like so many other ideas ahead of their time, Scheyer' s N.C. Lathe 
soon dropped from view. 


THE BIRTH OP THE NC ERA 

By 1948, the Parsons Corporation of Traverse City, Michigan, had 
developed a system for producing helicopter blade templates from numeri- 
cal coordinate data. By calculating airfoil coordinates on an IBM 602A 
multiplier and feeding these data points into a Swiss jig borer, the 
tedious task of laying out the job manually had been eliminated. In 
1948, John Parsons saw the drawings for a new design concept of integral- 
ly stiffened skins that had been developed by Engineers of the Lockheed 
Aircraft Corporation. In place of stringers, ribs and a skin being rivet 
ed together to form an assembly, the design called for single-unit struct 
ure consisting of a contoured outer surface with a series of pockets be- 
ing milled on the inner surface. The material left between the pockets 
would serve as the stiffening webs. After reviewing the design, Parsons 
conceived the idea of machining this part by reducing the design to a 
series of coordinate commands , feeding these coordinate commands onto 
a punched card, and controlling a milling machine by the punched cards. 
The Air Force became interested in this new concept of machining and, 
after evaluating demonstrations of a similar application at the Parsons 
Corp. and the Snyder Corporation in Detroit, awarded a contract on June 
15, 1949, to study its feasibility. From this beginning the NC Industry 
was born. 

In the Servomechanism Laboratories at MIT, a 28-inch Cincinnati 
Hydro-Tel milling machine was selected as the machine to be used in the 
study. The machine was extensively modified, including replacement of 
the table, cross-slide and head drives with variable speed hydraulic 
transmissions capable of being controlled by electrical impulses. By 
1953, sufficient data had been obtained to indicate the practical appli- 
cation of the NC approach. A detailed 24-page report on the results of 
this study appeared in the American Machinist Magazine on October 25, 

1954 and started a flurry of development activity within the industry. 
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By the end of 1955, a full compliment of NC machines, including 5- 
axis mills, drills, jig borers and lathes were on the market. Added in 
centive was given to the development of NC machines when the Air Force 
placed an order for 100 of the new contour-milling machines. Four com- 
panies were selected to build the machines and five companies were 
selected to build the readers. 



MILLING MACHINE SET UP IN SERVOMECHANISM LAB AT MIT 


With a full compliment of machines in the NC field, interest now 
turned to improvements. By 1957, automatic tool changing was offered 
by the Barnes Drill Company on one of their drills. IBM and Fosdick 
Machine Tool Company developed and introduced a tape-controlled jig-bor- 
er with an automatic tool changer in 1958. Also in 1958, Kerney & Treck- 
er developed a flexible automatic machining line for Hughes Aircraft. In 
the line, a drill, mill and jig bore were tied together and operated by 
a single, tape controlled reader. This unit was known as the Milwaukee- 
Matic Model I. 

The Milwaukee-Matic II was introduced later that year and made every- 
thing else look awkward. The work was mounted on an indexing table so 
that four sides of the part could be machined without relocation. A 
rotary drum mounted on the side of the machine held up to thirty differ- 
ent tools that would automatically be selected by tape command. By com- 
bining the functions of boring, drilling, milling, and tapping with the 
four-sided machining capability, the multifunction, or machining center 
was born. 

Initially, the NC machines received their commands from punched 
paper cards (IBM Cards) . This method was replaced before long, however, 
by the punched paper tape. Two variables made this move a logical one. 
First, the tape was understandable, convenient and worked relatively well 
in a hostile environment. Secondly, tape preparation equipment, such as 
the Friden Flexowriter was already in use and readily available, due to 
its demand in other business and accounting operations. As NC technology 
advanced, the taped programs became longer and the tape reader speeds 
kept going higher, thus creating a need for a more durable and reliable 
medium than that offered by the paper tape. The introduction of Mylar- 
based, or laminated tape satisfied this need for durability, but created 
another problem. Punching the Mylar tapes taxed the design limits of the 
flexowriter, resulting in increased maintenance for punch pin replace- 
ment and adjustments due to improper tape registration. 

ENTER THE COMPUTER 

The computer was first introduced into the market place as an account- 


ing tool. With an assist from the Air Force in the form of a develop- 
ment contract, this new science was expanded to include NC tape program- 
ming capabilities. Computer assisted programming was introduced and 
grew rapidly in the late 1960's and early 1970's with many companies 
offering computer time sharing and special programs to assist in NC tape 
preparation. 

As the state-of-the-art was advanced in computers, the flexowriter 
became obsolete as a business and accounting tool. The NC market was 
not yet large enough to justify continued production so it was discontin- 
ued. Microprocessor technology had been developed sufficiently by this 
time, however, to allow the creation of a smart terminal specifically 
designed and dedicated to the generation of NC tapes. Although improve- 
ments have been added, the basic design initially offered by companies 
such Numeridex is the most widely accepted method of NC tape prepara- 
tion . 


The first attempts at controlling the NC machine directly from the 
computer were made in the middle 1960's. However, the early vacuum tube 
computers were incapable of providing sufficient speed and versatility 
to make this application economical. DNC {Direct Computer Controlled 
N/C machining) gained a small measure of success only in a hand full of 
companies whose production rates justified it. 

As the computer technology advanced, aided by rapid advancements 
within the electronics industry, computers became smaller and less ex- 
pensive. Response times and data assimilation capabilities were also 
improved. By equipping a microprocessor with a memory and input/output 
circuitry, a microcomputer, small enough to include on a NC machine is 
obtained. Presently, this system of machine control is becoming the 
most prominent in the NC field. Known as CNC, the machine receives its 
coordinate commands from a program stored in its integral computer. 
Permanent storage of the machining programs is provided in a main, or 
parent computer and are fed to the machine upon demand. 

Appearing on the horizon is yet another generation of NC machine 
capable of writing its own programs, known as PNC (programmable numeri- 
cal Control) . The machine can be equipped to either punch a tape or 
write a computerized program while the first part is being machined by 
operator-controlled moves. Further, advantages include the ability to 
correct, or modify existing programs at the machine. 

COMPUTER GRAPHICS 

With the advent of computer graphic capabilities, the time required 
to develop and draw the design of a part has been reduced to a fraction 
of the time required before. By utilizing mathematical calculations 
previously programmed, the designer is able to complete his "drawings" 
on a scope and immediately send it to computer storage. When paper 
drawings (blueprints) are required, they may be obtained from a computer 
controlled printer. When change to the design become necessary, the 
design may be recalled to the scope, the changes made, and design re- 
entered into the computer's storage bank. Utilizing this system, there 
should be little chance for obsolete drawings to be present on the manu- 
facturing floor. 

Manufacturing planning activities also enjoy the benefit of in- 
creased accuracy and time savings by use of interactive graphics. The 
planner may call the design up on the screen and progressively plan 
the manufacturing steps in reverse order of fabrication^ Thus, he 
starts with a finished part and plans back to the starting material con- 
figuration) . As the machining sequences are being formulated, NC tapes 
(or programs) , holding fixtures and cutter configurations can also be 
designed and ordered. When the planner is finished, his input can all 
be stored in the computer memory bank. 

Production Control and scheduling activities have also been adapted 
to computer language quite successfully by a number of companies. With 
today's computer technology, it is quite conceivable to design, plan, 



schedule and manufacture parts without a single piece of paperwork 
being created. Other programs can be added to order material, accumu- 
late cost data and issue shipping instructions. 

THE ROLE OF QUALITY 

From the above discussion, it would appear that computer technology 
has provided almost all of the answers for an efficient manufacturing 
system. One factor, however, remains unresolved. This is the question 
of economical, yet efficient product assurance at the customer level. 
Consumer demands for product safety and reliability considerations have 
been augmented by government regulations and consumer advocate activities . 
This results in added pressure on the producer (and especially his Qual- 
ity Department) to take every reasonable precaution of assuring that 
only good units of product are offered for sale. 

Inspection (i.e. physical measurement of a finished part) is the 
traditional method of assuring that the parts passed on are within the 
limits of the design specifications. Before the introduction of NC, one 
inspector could normally inspect the work of eight machinists with a 
reasonable degree of accuracy. 

Today, one NC machine can perform the work of five to twelve con- 
ventional machines, while holding intricate dimensional relationships 
and tolerance never before attainable. Parts can be produced in a few 
hours that will require several hundred hours to measure when tradition- 
al layout methods are used. Of paramount importance, however, is the 
fact that inspection only sorts the good from the bad. It does not pre- 
vent the defect from occurring. 

Quality Control, in its ultimate form, assures that the product 
meets all previously established requirements and that the process is 
controlled within specififed limits. Again, however, the effectiveness 
of a quality control system depends upon its ability to measure the 
product in a timely manner. 

American Machinist Magazine describes the science of Quality Assur- 
ance as a predictive process. "To be effective, it requires assessment, 
monitoring, and control. Every process in the manufacturing cycle of 
a product must first be assessed to determine its potential for meeting 
previously established quality requirements. Once the process is select- 
ed, it must be monitored to make sure that the assessment of its potent- 
ial is still correct. Finally, the process must be controlled on the 
basis of such monitoring to keep the process under control." Under the 
terms of this treatise, every process in the manufacturing cycle must 
be assessed. This means that Quality must gain the expertise in the 
computer technology-related manufacturing practices, as well as convent- 
ional machining modes , in order to properly assess the manufacturing 
process currently in effect in the present hybred machine mix employed 
by most shops. As more and more computer technology is introduced, this 
requirement for computer-oriented Quality Engineers will become increas- 
ingly acute. 

In the last decade, the emphasis of computer technology has been 
applied to producing the part. Little effort has been made across the 
industry to provide equipment that will economically measure the product 
during selected stages of manufacture. As a result, each company has 
attempted to design their own inspection screens. This has often been 
accomplished by selective sampling, composite gaging, and reliance on 
the NC machine to produce a good part. Partially because of the lack 
of computer technology expertise and principally due to the lack of 
adequate funding, the quality discipline has lagged behind the acceler- 
ated advancement of its production counterpart. 

Quality Assurance, as a discipline, must re-examine its position of 
Product Assurance, especially in the area of machined parts. In high 
volume, continuous production environments, the use of sample inspections 
and control chart concepts have proven to be adequate for controlling the 



quality of the end product. Process controls have been demonstrated to 
be successful in the control of heat treatment and chemical process oper- 
ations. First part inspections, combined with sampling of the finished 
lot is also a popular means of control. Although these methods have 
been proven to be successful in the past, their effectiveness in con- 
trolling the machining concepts of tomorrow will, at best, be marginal. 
Consider for a moment these facts: 

o Lot sizes are now less than 50 pieces for 75% of 
all production runs. 

o 90% of all NC machine programs take less than 
27 minutes to run. 48% are run in less than 
10 minutes. 

o Rework of part nonconformities are almost non- 
existent on most NC machines. 

Once these facts are examined, in the light of today's Product Assurance 
programs, it is immediately apparent that additional expertise and meas- 
urment capabilities must be obtained. Otherwise, prohibitive scrap and 
rework costs and excessive inspection bottlenecks will impede the progress 
of improved machining productivity. The other alternative of relying on 
the machine's ability to produce only good parts could prove disasterous 
in light of today's product liability litigations. 

A successful quality assurance program for the CAD-CAM concept must 
begin at the earliest stage of the program — the design phase. Decis- 
ions must be made as to how and where the product will be measured for 
compliance. Inaccessible dimensional relationships and specification 
contradition should be resolved before the manufacturing effort is in- 
itiated. Additional measurement equipment and programs that will be re- 
quired to verify the product can be ordered in advance to assure its 
availability, when needed. In many companies, modified forms of this 
program are currently in effect. Most of the evaluations, however, are 
currently made from a formal blueprint. 

In the CAD/CAM era, two new factors must be considered. The blue- 
print will be a video protrayal on a scope and the machining will be 
performed in a fraction of the conventional machining time on a computer 
controlled machine. In order to perform an adequate assessment of the 
design and develop an effective, yet economical verification program, 
the Quality man must have sufficient expertise in both computer technology 
and shop practices. 

Since operator compensation cannot be made for such variables as 
worn lead screws, slack bearings and cutter wear on a NC machine, mainten- 
ance and calibration controls are of paramount importance. From experi- 
ence, we have proven that the desired matalurgical properties can be ob- 
tained during heat treatments if we adequately control the furnace and 
sample the lot of material. These same results can reasonably be expect- 
ed from an NC machine if adequate evaluations are scheduled in a timely 
manner. Several factors must be considered in designing such a program. 
Among the more prevelant areas are the following: 

o Machine Installation : The initial machine installa- 
tion, levelling and adjustments made must be verified. 

The use of a laser interferometer to verify the accuracy 
at a reasonable cost. 

o Periodic Calibration : Periodic calibration checks on 

a machine will assure that the machine is still capable 
of maintaining the tolerance allowables of a design. 

The use of test programs to make a test part with given 
dimensions approximating all of the machine moves have 
been used quite successfully. Position checks measured 
by laser interferometer have also proven adequate. 



o Tooling and Cu tt er Contro l: Quality verification of 
tool holder settings, tool regrind and holding fix- 
tures will provide an additional degree of assur- 
ance. Cutter wear studies provide for changing the 
cutting tool before a nonconformity appears. 

Automated inspection equipment is beginning to appear on the market. 
Computerized coordinate measuring systems are now available that can pro- 
vide a wealth of analyzed data. Non-contact gaging concepts are being 
pursued by many companies, among the more notable being United Technol- 
ogies Research Center (laser diodes) , EMR Photo-Electric ( Comp Gage 
System) , and Jones and Lamson (Metric Eye) . Total system quality control 
is the goal of Lockheed Missile & Space Company's IPQC System. The ulr 
timate selection of the equipment best suited for each company must be 
made by the Quality Department. Since each piece of this equipment is 
expensive and unique, decisions must be based on application, operator 
familiarization, maintenance and economic considerations. 

To further compound the situation are two new innovations: Auto- 
matic Adaptive control of machine tools and Group Technology. Under the 
Adaptive Control concept, the workpiece will be continually monitored 
and the necessary adjustments to maintain the required accuracy at opti- 
mum machining rates are automatically made. Group technology consists 
of grouping families of similar parts so that they can be produced more 
efficiently. Quality Assurance inputs during the definition and design 
stages of both of these programs are necessary to insure a successful 
implementation . 

The building blocks for tomorrow's quality programs must begin to 
be carefully selected today if we are to be successful tomorrow. This 
must begin with personnel, not only fully knowledgeable of today's re- 
quirements, but also with sufficient knowledge of computer technology- 
related systems currently under development to maintain pace with manu- 
facturing advancements. Old concepts must be re-evaluated and adjust- 
ments be made accordingly. 

The computer has provided us a tool with which we can produce goods 
and services at a fraction of the time previously required. To assure 
that these goods and services are continually produced in a prescribed 
manner requires a system of controls and quality assessments. In order 
to design and maintain this system of controls and quality assessments, 
a knowledge of the computer technology-related manufacturing principles 
must be present. 

DOES YOUR QUALITY DEPARTMENT HAVE THIS CAPABILITY ? 
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Frank J. Sindelar 
IBM Corp. (Retired) 
Poughkeepsie, New York 


ABSTRACT 

The Standard Committee serves to assure an interface with standard 
bodies external to ASQC and to establish liaison within ASQC with the 
Division and Technical Committees, who provide the expertise and man- 
power required for the design, development and support of an effective 
standard program. 


INTRODUCTION 

Accreditation of ASQC as a standards writing organization was 
announced by the American National Standards Institute on March 14, 
1977. This event, while significant in so far as ASQC is concerned, is 
"just the tip of the iceberg" in terms of the preparatory activities 
which preceded accreditation. 

ASQC management became concerned and involved in society standards 
activities and direction in 1973 , and in recognition of this attention 
initiated a program of redefinition and change. In broad terms, what 
was being sought was an expanded participation by the society and its 
members into a leadership role in the development and promulgation of 
standards dealing with the assurance sciences. 

During this period of evolution, ASQC assumed the responsibility 
in 197*1 for the Secretariat to the ANSI-Z1 Committee on Quality 
Assurance. This assignment placed an increased requirement on Society 
management for the establishment and initiation of an ASQC Standards 
program, which would assure an acceptable organization with high order 
professional and technical participation in the standards development 
arena. (See Figure 1.) 
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A redef inition of the ASQC Standards Committee was undertaken with 
the following requirements in mind: 

a) Well defined ASQC policies, 

b) An effective Standards Committee organization 

c) An organization involving all interested ASQC Divisions and 
Technical Committees 

d) Procedures which would assure the Society and its members of 
professional visibility in the development and release of both 
ASQC and American National Standards 

In view of the above, the ASQC Standards Committee has been 
organized as shown in Figure 2. 



FIGURE 2. 

ASQC STANDARDS ORGANIZATION 


It should be noted that the Standards Committee has retained much 
of what was already in place and has added the functions of the 
Standards Review Board (SRB) and a formal representation of Divisions 
and Technical Committees by "Standards Representatives". 

The Standards Committee , made up of members of the reporting 
groups, is responsible for management of the standards program in ASQC. 
Policies and Procedures, G-10, dated November 18, 1977, is the current 
management document which provides overall direction to the effort. 

The Mandatory Standards Group is responsible for liaison with 
appropriate agencies of government that are producing mandatory 
standards that affect the quality and reliability discipline, and to 
provide a communicating link with technical units of the Society for 
appropriate technical advice and input to those agencies. 

The Standards Review Board (SRB) is responsible for assuring that 
writing groups who have developed standards, have infact, operated in 
accordance with designated procedures in establishing the required 
"National Consensus" when declaring either an ASQC Standard or an 
American National Standard. 






The Coordination and Procedures Group , working in consonance with 
the ASQC technical staff and an associated Standards Document Control 
Center is responsible for assuring that appropriate procedures exist 
for both the overall operation of the Standards Program, as well as the 
control documentation required for the development, publication, and 
maintenance of the standards documents. 

The Standards Group is responsible for assuring and maintaining 
the overall technical competance of the Society Program. It is also 
responsible for liaison and interface with the Division and Technical 
Committee Standards Representatives because of program dependence on 
these units as the source of technical expertise in the development 
and maintenance of standards documents. 

As we noted previously, the involvement of the Division and 
Technical Committees in the Societies Standards Program is of 
paramount importance. The success of any standard document depends on 
a need, and this need can be best detected by professionals in the many 
industrial and service areas actively involved with the assurance 
sciences . 

Members of the several division and technical committees are in a 
position to access such needs and, furthermore, transmit same into the 
ASQC Standards Program for evaluation and action. Therefore, it 
follows that strong standards initiating actions will originate with 
standards committees in these Divisions and Technical Committees. 

The Society Standards Committee has a responsibility for assuring 
that we have a mechanism (both management and technical) which is 
capable of assisting in the prioritizing, and development, and in 
assuring that needed standards proposals which are entered into the 
system meet technical and procedural requirements and ultimately become 
available to users in the shortest possible time. 

Division and Technical Committee Standards Representatives have 
a key role in assuring strong liaison between their respective D.T/C 
Standards Committees and the Society Standards Committee, This is a 
two way communications link since the overall concern is one of meeting 
an industrial or national need with precision and timeliness. As of 
January, 1978, we have 18 D.T/C Standards Representatives announced. 
(See Figure 3») 
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FIGURE 3. 

ASQC -STANDARDS INTERFACE 




It was recognized, early in the committee study, that in all 
probability there would be a significant increase in the number of 
persons involved in various phases of the standards program throughout 
ASQC . In order to assure that all participants receive the required 
operating guidance, the committee has published an "Operations Manual". 
The Coordination and Procedures Group is responsible for content and 
revision. The following is an outline of manual contents: 

1.0 ASQ.C Standards Committee 

Appropriate organization charts and administrative procedures 
for each of the Standards Groups and the Review Board. 

2.0 Division and Technical Committee Representatives 
Interface requirements for representatives within their 
respective division and technical committee standards 
operations, and in liaison with the Standards Committee. 

3.0 Standards Development and Management Procedures 

Seven operating procedures specific to the management of a 
standards project, as well as the technical requirements 
regarding document content and control. 

4.0 ASQ.C/ANSI-Z1 Secretariat 

Charges , rules and regulations for the operation of this 
committee . 

5.0 Appendix "A" 

Key ASQC management Policies and Procedures affecting the 
standards program. Also, the Standards Committee long range 
plan. 

6.0 Appendix 11 B" 

ANSI organization charts, and procedures. Also, the ANSI 
Style Manual outlining format requirements for an American 
National Standard. 

What we have covered up to this point is a review and update of 
some of the past work done by our society in bringing the Standards 
Program in step with todays marketplace. 

In the past, ASQC members and staff have been active participants 
along with members of such societies as ASTM , IEEE, SAE, ASME, UL and 
NEMA in the development of standards. 

However, with accreditation we assume the responsibilities of 
initiation and authorship and including development, publication, and 
maintenance of standards document. 

A review procedure, including detailed documentation of both the 
reviewers and writing group comments and/or action is a mandatory 
requirement as a part of pre -publication. Evidence of these data is 
under the control of both the ASQC Standards Review Board and the ANSI 
Board of Standards Review. 



You have probably seen announcements in Quality Progress similar 
to this one which appeared in the December, 1977 issue. National 
consensus procedures require broad public exposure in order to meet 
ANSI requirements for comments during the development/draft stages of 
a voluntary standard. (See Figure 4.) 


ANNOUNCEMENT 

Revision to ASQC Standard A2-1071 
(ANSI Standard Zl. 6-1971 ) 

"Symbols and Definitions for Acceptance Sampling 
Involving Percent or Proportion Variant Units in 
a Lot Batch" 

IS NOW AVAILABLE FOR COMMENT 

This revision is a major expansion in revised 
format of the second of three definitions standards 
published by ASQC. The revision contains 62 general 
sampling terms : 26 terms related to attribute sam- 

pling, 4 indices for acceptance sampling, and 32 
terms related to variables sampling. The expansion 
covers the major sampling plan types for both 
attributes and variables techniques , including 
single, double, multiple, sequential, chain, skip- 
lot, continuous, and isolated lot. 

The standard is available by sending a pre- 
paid check or money order to: 

The American Society for Quality Control 
Attention: R. L. Griffith 
1 6 1 W. Wisconsin Avenue 
Milwaukee, Wisconsin 53203 

Single copy price: $4.00 

COMMENT CLOSING DATE 
FEBRUARY 15, 1978 


FIGURE 4. 

(Appeared in December 1977 
issue of Quality Progress) 

In view of the extraordinary coverage provided by 26 divisions 
and technical committees, ASQC members are in an excellent position 
to advance standardization in the assurance sciences as applied to 
a broad spectrum of processes, products, and services. 

In the context of Industrial, National and International 
standardization, ASQC professionals who are interested in standards 
development and application will find broad opportunities for involve- 
ment in either the technical, or management aspects of standards 
operation. (See Figure 5*) 
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ANSI -INTERNATIONAL INTERFACE 


The ASQC Standards Program has the capability for assuring an 
effective involvement of members and staff in meeting ANSI requirements, 
with documents which support Industrial, National and International 
user needs . 
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ASQC/ANSI DEFINITIONS STANDARDS - A STATUS REPORT 


Richard A. Freund 
Management Services Division 
Eastman Kodak Company 
Rochester, New York 14650 


The ASQC Standards Committee was established in 1947 with the charge 
to "... study and select symbols, concepts, terms, procedures, and other 
matters which it feels the Society might standardize to advantage 
The first standard it developed was Al-1951, "Definitions and Symbols for 
Control Charts". This was updated with slight modifications, primarily 
with respect to notation, in a 1969 revision which was adopted by ANSI in 
1971 and issued as ASQC : Al-19 7 1 , ANSI : Zl. 5-1971. 

In 1957, a second definitions standard, A2-1957, "Definitions and 
Symbols for Acceptance Sampling by Attributes", was prepared. This was 
revised in 1962 as A2-1962 and again in 1971, primarily with respect to 
clarification and updating of comments associated with the original 
terms, and in 1971 it was adopted by ANSI and issued as ASQC:A2-1971, 

ANSI :Z1. 6-1971. 

A third definitions standard, A3-1964, "Glossary of General Terms 
Used in Quality Control" was issued in 1964. In 1971 this standard was 
enlarged to include some of the more general terms dealing with control 
charts and sampling, and some of the original terms were modified slight- 
ly. This revision was adopted by ANSI and issued as ASQC : A3-1971 , ANSI: 
Zl. 7-1971. 

In early 1975, the ANSI Zl Committee on Quality Assurance reviewed 
the overall needs for quality definitions standards. Almost concurrent- 
ly, the ASQC Standards Committee decided that the time was appropriate 
for a major updating of all three of these ASQC standards. As a member 
of the Standards Committee, I was named chairman of a writing committee 
to accomplish this task. The Statistics Technical Committee was asked 
to recommend volunteers who would serve with me on this task group. For 
a little over two years now, the writing committee consisting of: T. W. 
Calvin, J. W. Foster, R. A. Freund, J. D. Hromi, J. S. Hunter, N. L. 
Johnson, J. V. Lavery, W. M. Mead, and H. M. Wadsworth has proceeded to 
extensively increase the number of terms defined, and to consolidate and 
unify the concepts and definitions included in the three original stan- 
dards. The scopes of these standards were reevaluated in order that they 
would be consistent with the overall strategy proposed by the ANSI Zl 
Committee. 

It was decided that the scope of the A1 Standard should be expanded 
beyond the Shewhart X and R and attribute charts that had been well 
treated in the original. Other more recent and widely used control chart 
techniques were to be included. For example, the Acceptance Control 
Chart, the Exponentially Weighted Moving Average Chart and the Cumulative 
Sum Chart have been included in the new revision. Furthermore, it was 
decided that this revision should attempt to show how these techniques 
relate to each other, and to indicate when each might be used most effec- 
tively . 

The A2 Standard was to be enlarged to include variables sampling 
plans as well as those for attributes. In addition, we wanted to show 
the relationships among the different types of plans such as those for 
continuous processes with lots, continuous processes without lots, and 
isolated lots. We also wanted to contrast the different indexing systems 
such as AQL, LQL, AOQL and minimum cost to assist in the understanding of 
their selection. 

Next, the A3 Standard was to be cleared of those terms that more 
properly belonged in the Al and A2 Standards so that it could concentrate 
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on a more extensive effort in the area of quality systems terminology. 


Beyond these three standards, the ANSI Z1 Committee envisions docu- 
ments on reliability, sampling in a general rather than acceptance 
sense, design of experiments, and an alphabetical glossary that does not 
involve the comparative comments of the detailed standards, but summari- 
zes the full field of quality related terminology. Work is already under 
way on these. 

Now let us look at the Al, A2 , and A3 Standards in a little more de- 
tail. We are viewing the prime purpose of standards as a means of im- 
proving communications through consistency. A major source of difficulty 
in either the technical or non-technical fields is that a word or phrase 
is interpreted differently by various people, depending on their back- 
ground and experience. George Bernard Shaw observed that "England and 
America are two countries separated by the same language". In our pro- 
fession, as in others, we often use identical words, but they may mean 
quite different things. Sometimes this is a matter of general usage ver- 
sus specialized technical usage, but quite often it represents an evolu- 
tion in application. 

Quality Control should be of interest to all kinds of manufacturing, 
processing, and service endeavors dependent on almost the entire gamut of 
the arts and sciences. With the large diversity of backgrounds involved, 
it is little wonder that our common language has been interpreted in so 
many ways. For this reason, we have chosen in these standards to make 
extensive use of explanatory and comparative comments and notes. This 
has lengthened the document, and perhaps made it a little less convenient 
to skim. However we felt it has added an important dimension to the use- 
fulness of the standards. At the same time, we tried to resist the temp- 
tation to make these standards into textbooks or detailed "how to" expla- 
nations. The Statistics Technical Committee and other groups are working 
on that type of effort. There are a few technique oriented standards 
such as the Bl, B2 and B3 series on Shewhart-type control chart construc- 
tion and application, which now need updating, and there are plans under 
consideration for others of this nature. The function of the definitions 
standards, however, is to provide the common concepts for those who apply 
or explain the techniques and not to provide the detailed application ex- 
amples . 

One area that had to be tackled in dealing with control charts and 
acceptance sampling was the terminology associated with events that are 
being reported. Traditionally, the early textbooks, papers and applica- 
tions dealt with a simplistic approach to language wherein words like 
"defect" or "defective" merely meant that something was different and was 
to be detected. Often these words were intended to indicate poor prod- 
uct, but many times the words were inappropriately used, although the 
true meaning was understood within the narrow confines of their specific 
application. For example, in recent CPSC hearings on sampling, concern 
was expressed about sampling plans with high AQL values described in 
terms of a relatively sizable percent of defective product. The meaning 
attached to "defect" by the groups raising the question was that the 
product would be unsafe or would not work in the hands of the consumer. 

In general, the sampling plans were intended to be used in terms of con- 
formance to specifications which often were tightened in order to provide 
protection against such aspects. That is, the specifications were not 
given simply in terms of " f itness-for-use" . Imprecise language was help- 
ing to create a significant barrier to understanding. 

This subject has been widely discussed in the literature for many 
years, and in the February 1977 issue of QUALITY PROGRESS a report entitled 
"Saying What You Mean to Say" described the work being done by the writ- 
ing committee in an attempt to clarify this confusion. Section 14 of the 
new ANSI/ASQC Standard Al-1978, "Definitions, Symbols, Formulas and Ta- 
bles for Control Charts", includes the definitions of additional terms 
which, if used consistently, should help clarify this issue. This is not 
a word substitution game. "Defect" and "defective" are still part of the 
language, but are specific to " f itness-for-use" considerations. Confor- 
mance to specifications involves terms like "nonconforming units" or 



"nonconformities". Most acceptance sampling really involves comparison 
with specification requirements, and while " f itness-for-use" must be con- 
sidered in setting those specifications, it is not a direct relationship. 
Another term, "imperfection", has been introduced to recognize that some- 
thing has deviated from some target without attaching judgment criteria 
as to whether specifications are satisfied or product meets the "fitness- 
for-use" criteria. This is important in control operations where trends 
give advance warning so that problems can be prevented. 

Sections 1-3 of the ANSI/ASQC Standard Al-1978, "Definitions, Sym- 
bols, Formulas and Tables for Control Charts", deal with the general as- 
pects of the Shewhart Control Charts, Acceptance Control Charts and Adap- 
tive Control Charts. Sections 4-11 treat the specific variations in 
these charts, with section 12 defining general terms. Section 13 defines 
those terms related specifically to control charts for variables, includ- 
ing factors and formulas, and section 14 similarly treats those for at- 
tributes charts. Sections 15-17 deal with terms and formulas specifical- 
ly related to Acceptance Control, Exponentially Weighted Moving Average 
and Cumulative Sum Charts, respectively. 

The A2 Standard, "Terms, Symbols and Definitions for Acceptance Sam- 
pling Involving the Percent or Proportion of Variant Units in a Lot or 
Batch", is divided so that Section 1 covers general acceptance sampling 
terminology. Sections 2 and 3 are specifically related to attributes 
sampling. Section 4 deals with the types of indices. Sections 5 and 6 
relate to variables sampling. 

The A3 Standard, "Quality Systems Terminology", defines such basic 
terms as quality and reliability, quality systems, quality programs, 
quality assurance, quality control, quality engineering, reliability en- 
gineering, quality audits and related terms. As in the other two stan- 
dards, comments and notes are used extensively to describe relationships 
among these functions. 

In summary, we have made considerable progress in describing and de- 
fining some of the fundamental terminology and concepts for the quality 
profession. Work is continuing now on the additional important areas 
that fit into the overall goals of the ANSI Z1 Committee on Quality As- 
surance. 
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STATUS REPORT OF Z1.15 SYSTEMS STANDARD 


Robert W. Peach 

Manager, Quality Assurance Engineering 
Sears, Roebuck and Co. 

Chicago, IL 60684 

In 1973, the American National Standards Institute asked ASOC to assume the 
Secretariat of a new Quality Assurance Committee. One reason for the need of such 
a committee was the fact that many new product standards were being written con- 
taining Quality Assurance provisions. Standards writers recognize that not only 
must a product standard specify criteria to be met by the end product, but the 
means of accomplishing this in mass production must be considered. When the ANSI 
Z-l Quality Assurance Committee was formed, it was understood that there was an 
urgent need to develop a generic Duality Systems Standard that would be usable 
either directly, in modified form, or as a reference, for inclusion in product 
standards . 

At about the same time five years ago, a group of Quality Assurance Managers 
working in consumer products industries were already engaged in an effort to prepare 
a Quality Assurance Systems document that would be applicable to consumer products. 
Our concern at that time was that the existing quality systems documents had been 
developed for use in defense, aerospace and large commercial installations, but were 
not directly applicable to mass-produced consumer products, and applicable or not, 
were in fact seldom used in consumer product lines. We were looking towards the 
development of a quality systems standard that would be used in industries in which 
we worked. Shortly the Consumer Products Committee found itself aiding in systems 
standard development, and taking part in the Z-l Committee effort. 

Two years ago, at the 30th annual Technical Conference of ASQC in Toronto, 
Canada, we discussed the status of our work in the preparation of the generic 
Quality Systems Standard. As reference, some of the material presented at that 
time is included as background to today's presentation. 

Rather than first searching out the many product standards which include 
various Quality Assurance provisions, the ANSI Quality Assurance Sub-Committee 
that addressed the problem of developing a generic Quality Systems Standard took 
the existing umbrella standards which came to its attention, and outlined them to 
establish their scope and content. Table I lists an initial group of such stan- 
dards, British and Canadian documents, and various other works containing different 
degrees of technical depth, and using various approaches to describe Quality 
Assurance systems. Each of the first group of standards were outlined by the 
Committee to identify the subject areas covered. Tables II and III are outlines 
of two of these standards. Next, a matrix was prepared to compare the contents of 
the standards to each other. While incomplete, and containing some inaccuracies, 
the matrix does demonstrate that not only does the content of the various documents 
vary considerably, as would be expected, but no one document seems to define all 
significant areas of a Quality Assurance system. 

The Committee task was to construct a new outline, covering all elements of 
Quality Assurance systems which seemed significant. This was done by first listing 
all of the outlined subjects on 3 x 5 cards, and sorting until a logical and workable 
order was achieved. This initial sorting demanded certain working decisions which 
had an influence on the end result. For example, while elements of a quality system 
such as receiving inspection, in-process inspection, or final inspection are 
separately definable, the subject of the use of statistical methodology might be 
included under each of the various inspection elements, or might be presented in a 
single section and cross-referenced. This latter course was chosen, somewhat 
arbitrarily, in order to avoid repetition and to reduce the size of the final 
document. 

When a "complete" outline was developed, portions of the outline were 
assigned to writing teams for "fleshing out". To do this, since the content of the 
original input document had been cross-referenced, copies of each of the sections 
of the input documents were given to each of the section writing teams. While 
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they worked with the original outline, they had access to the content of the other 
documents and were encouraged to be guided by these documents. 

During the working sessions of Committee efforts to develop the document, 
questions came up as to the objectives as well as the mechanics of accomplishing 
the writing of a usable generic document. It was soon recognized that, because 
of its scope, not all products or industries would use all of the elements of the 
generic Standard. Thus, as a generic document, the Standard would be used as a 
basis for developing specific documents for individual product lines and industries. 

In working with the generic guidelines, the user must realize that the degree 
of control necessary for a specific product or production situation will vary, and 
should be reflected in the Standard, as modified. The thoroughness required for 
Nuclear power Standards is inappropriate for shoe manufacture Quality system 
requirements related to user safety for television sets place demands far beyond 
those required for bedspread manufacture, even though the basic principles upon 
which the control systems are based are remarkably similar. 

A perishable food item may actually require all elements of a Quality 
Assurance system, but not all details of each element will apply. The Standard 
discusses installation, service and use in the section of Field Performance. While 
acceptable performance to the user is at least as essential for a food product as 
with any other product, food does not require assembly instructions or spare parts. 
Consequently, items such as these must be reviewed for appropriateness to the 
product, and rewritten so that they will directly apply. In contrast, a complex 
consumer product, such as an electronic device, might well require application of 
all elements (chapters) of the Standard, from planning and design assurance, through 
field performance and corrective action. Indeed, portions of the Appendix (that 
are technically not a part of the Standard) might also be included as a part of the 
Standard for an electronic item, because of their relative importance. 

The question was posed as to why such a Standard is needed, when so many other 
documents are available? The wisdom of our answer will be proven out at a later 
date, when it is established whether the generic Standard does serve a purpose to 
industry. As noted in the matrix, possible shortcomings of most of the existing 
quality systems documents were in one or two areas. Some do not provide for 
adequate system pre-planning; a second shortcoming was often the lack of inclusion 
of follow-through of the product to the user, and the feedback of usage information 
back to the manufacturer. And even where these were included, they often were not 
covered in proportion to their importance in today's industrial climate. 

Within this past year, after considerable deliberation by Committee members, 
the name of the document was changed to "Generic Guidelines for Quality Systems." 
This change was made when it became evident that without it the use of the document 
could be misunderstood. This change strikes at the heart of the intent and purpose 
of the Standard. The Foreword, Statement of Scope and Application sections of the 
document clearly state the intended usage. However, without the word "Guidelines" 
in the title, there remained the very real danger that the use of the Standard, 
in toto, and without modification, would be dictated for use in cases where only a 
portion of the document should actually be put into use, or where modifications 
were essential. 

The end result of the work of developing the Generic Quality Systems Guidelines 
is a broad document that we believe includes all significant elements of Duality 
Assurance from design concept through end usage, feedback and extended use in 
service. It is intended that all are in logical balance. The Guideline is a 
framework against which individual product guidelines can be developed following 
the same structure or format. It provides an expression of the basic elements 
of Quality Assurance that can be applied in essentially all areas of manufacture of 
commercial, industrial and consumer products. It is hoped that its existence will 
now aid major areas of American and world industry in understanding the basics of 
Quality Assurance. A list of Committee members is attached. However, it is 
impossible to list all who have made valuable contribution to the Standard. 


Briefly, here are the basic sections of the document: 

1 . Foreword and Scope . The abstract of the document states " Generic 
Guidelines for Quality Systems covers and briefly describes the 
significant elements that should be considered in the quality system 
of a manufactured product. This embraces quality policy, design 
assurance, quality control at various stages of production, field 
performance and product liability. The Standard is intended to 
provide Generic guidelines for a product quality system. As such 

it is not directly applicable to individual products or industries, 
but must be tailored to the specific case, making use of each element 
that is applicable." 

2. Application. This section stresses the advisory nature of the 
guidelines, and the need for continually updating a quality system 
so that it will not stagnate. 

3. Definitions of Terms Used in Quality Systems . The list of terms 
defined in the Standard is brief, since longer lists of quality 
terminology are available in existing ANSI/ASOC Standards Al, A2 and 
A3 and elsewhere. Definitions in the Guidelines document are limited 
to basic concept terms such as "Ouality of Design", "Quality Systems", 
"Quality Control" and "Quality Assurance." 

4. Policy Planning and Administration . This section calls for a written 
Quality Policy, including objectives and an organized approach for 
carrying them out. 

5. Design Assurance provides for design validation and re-qualification, 
plus design change (Configuration) control. 

6. Control of Purchased Materials includes: 

Review of supplier quality capability 
Supplier quality information 
Quality Control or surveillance at source 
Receiving inspection, and 
Identification of purchased materials 

7 . Production Quality Control provides for : 

Planning and controlling the process 
Completed item inspection 
Handling, storage and shipping 
Product identification 

Development and use of quality information 

8. User Contact and Field Performance includes: 

Statements of product objectives 
Review of advertising information 

Use of sales and service organization quality information 
Control of installation, service and use, and 
User/consumer feedback 

9. Corrective Action includes: 

Procedures for detecting and reporting problems 
Evaluating and follow-through to correction 

The Appendix includes: 

Sampling and Other Statistical Tools 
Employe Selection and Motivation 
Measurement Control of the Inspection Process 
Product Liability and User Safety 
Economics of Duality Control 
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As the Standard is actually put into use, the ANSI Z-l writing group has several 
alternative new writing tasks to consider. It intends to aid in the application of 
the Standard to specific industries or products. It is considering the writing of a 
"How to do it" supplement, that will expand upon each of the sections. (Actually, 
much of this is available as a result of Committee work already completed.) It also 
has been suggested that a more concise version of the Standard be prepared for wide 
distribution in industries where Quality Control techniques are less well understood. 
Any or all of these various efforts may be embarked upon. But most important at this 
time is the effective application of the Z 1.15 Standard in places where it is needed 
and can be effectively used. To do that requires careful, professional application of 
the Standard by those who understand the principles upon which it is based. Only 
then can it become a worthwhile contribution to the Quality Control profession. 


MEMBERS OF WRITING COMMITTEE 
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TABLE I 


SELECTED BOOKLETS AND PUBLICATIONS 
DESCRIBING QUALITY ASSURANCE PROGRAMS 


U. S. Organizations - General 

Specification of General Requirements 
for Quality Program 

Procurement Ouality Control 
A Handbook of Recommended Practices 

Practice for Certification by 
Producer or Supplier 

Quality Program Requirements 

Evaluation of a Contractor's Ouality 
Program 

Inspection System Requirements 

Evaluation of a Contractor's 
Inspection System 

Supplier Quality Assurance Program 
Requirements 

Safety in the Marketplace 
(Orange Cover) 

Product Performance and Servicing 
(Green Cover) 

A Tested System for Achieving 
Quality Control 

A proven Method of Quality Control 
Product Safety 

CANADIAN/BRITISH BOOKLETS 
Quality Assurance 

A Guide to Quality Assurance 

Quality Assurance Requirements for 
Industry 

SPECIFIC INDUSTRIES OR PRODUCTS 

Quality Assurance-Criteria for Nuclear 
Power Plants 

Requirements for Electronic 
Component Parts Manufacturers 


ASOC Standard C-l 
ANSI Std. Z1.8 - 1971 

ASOC Vendor - Vendee Technical 
Committee - 1973 

ANSI Z-34.2 - 1969 


MIL-Q-9858A 16 December 1963 

Handbook H-50 - 31 October 
1960 

MIL-I-45208A - 16 December 
1963 

Handbook H-51 - 3 January 
1967 

MIL-STD-1535 (USAF) - 
1 December 1972 

National Business Council 

for Consumer Affairs (April 1973) 

National Business Council for 
Consumer Affairs - September 1973 

SBA Technical Aids No. 91 - 
January 1969 

Research Institute 
Recommendations 9-9-66 

Employers Insurance of Wausau 
Symposium - June 1968 


Canadian Standards Association 
April 1973 

BS 4891:1972 - British 
Standards Institute 

British Standards Institute 
73/85038 - May 1973 


Appendix B to 10CFR50 - 
June 1970 

(1) Inspection System 
Requirements 

(2) Ouality Program Requirements 
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TABLE 11 ~ C - 1 . FICAT1 0 N QF__GENERAL requirements for a quality program 
asqc standard "cT“ — — — — — 


Definition of Terms 

1.1 Quality Program 

1 . 2 Contractor 

1 . 3 Buyer 

1.4 Inspection 

Scope 

1 Applicability 

2.2 General Expense 

Requirements 

3* 1 Quality Management 

3.1.1 General 

3.1.2 Organization 

3.1.3 Procedures 

3.2 Design Information 

3.2.1 General 

3.2.2 Change Control 


3. Continued 

3.4 Material Control 

3 . 5 Manufacturers 

3.5.1 General 

3.5.2 Process Control 

3.5.3 Special Processes 

3 . 6 Acceptance 

3.6.1 General 

3.6.2 Sampling Inspection 

3.6.3 Nonconforming Material 

3.7 Measuring Instruments 

3.8 Quality Information 

3.8.1 General 

3.8.2 Quality Control Records 

3.8.3 Corrective Action 

4. Quality Program Audits 


3.3 Procurement 


3.3.1 General 

3.3.2 Source Information 

3.3.3 Fabricated Material 

3.3.4 Raw Materials 


1 . 


2 . 


3. 


4. 


5 . 


TABLE III QUALITY PROGRAM REQUIREMENTS MIL-Q-9858A 


Scope 

iVl Applicability 

1.2 Contractual Intent 

1 . 3 Summary 

1.4 Relation to Other Contract 

Requirements 

1.5 Relation to MIL-45208 

Superceding, Supplementation S 

Ordering 

271 Applicable Documents 

2.2 Amendments and Revisions 

2.3 Ordering Government Documents 

Quality Program Management 

3.1 Organization 

3.2 Initial Quality Planning 

3.3 Work Instructions 

3.4 Records 

3.5 Correction Action 

3.6 Cost Related to Quality 

Facilities and Standards 

4.1 Drawings, Documentation and 

Changes 

4.2 Measuring and Testing Equip- 

ment 

4.3 Production Tooling Used as 

Media for Inspection 

4.4 Use of Contractor's Inspection 

Equipment 

4.5 Advanced Metrology Requirements 
Control of Purchases 

5.1 Responsibility” 

5.2 Purchasing Data 


6. Manufacturing Control 

6.1 Materials and Materials Control 

6.2 Production Processing and 

Fabrication 

6.3 Completed Item Inspection and 

Testing 

6.4 Handling, Storage and Delivery 

6.5 Nonconforming Material 

6.6 Statistical Quality Control and 

Analysis 

6.7 Indication of Inspection Status 

7* Coordinated Government/Contrac tor 
Actions —-*• 

7.1 Government Inspection at sub- 

contractor or Vendor facilities 

7 . 2 Government Property 

7.2.1 Government- furnished 

Material 

7.2.2 Damaged Government- 

furnished Material 

7.2.3 Bailed Property 

8 . Notes 

8.1 Intended Use 

8.2 Exemptions 

8 . 3 Order Data 


SYSTEM RELIABILITY ANALYSIS: A TUTORIAL 


Mitchell 0. Locks 
Professor of Management Science 
Oklahoma State University 
Stillwater, OK 74074 


INTRODUCTION 

This paper deals with the logical formulation of a system for purposes of relia- 
bility analyses and both exact and approximate methods of calculating the system 
reliability. The first part deals with the logical concepts, and the second part with 
probability calculations. 

The logical formulation in Part 1 starts from first principles. The universal 
set U of system states is a "Boolean algebra". The "power set" & is the set of subsets 
of U. The subset of system success states or "paths" is a "lattice" within U» also an 

element of v 3 , represented by a Boolean polynomial. The terms of the polynomial are 

monomials which give the indicators for the "sublattices" or "complete subsets” of U 
within the lattice of paths. The optimal representation of this lattice polynomial is 
in the minimalized form; the terms of the minimalized lattice of paths are called the 
"minimal paths". Similarly, the subset of system failure states or "cuts” is a lattice 

polynomial whose terms are sublattices within the lattice of cuts; the terms of the 

minimalized lattice polynomial are the "minimal cuts". Logically consistent systems 
(also known as "coherent structures") have certain properties with respect to the 
partial ordering of paths and cuts. The concepts of Boolean logic, minimalization, 
etc., apply to systems that do not have the "consistency" property, as well as to those 
that do have it. 

The second part of this paper deals with ways of using a minimalized lattice poly- 
nomial to derive a numerical-valued probability function from which to calculate the 
system reliability: also computer software, error bounds and approximations. The 
reliability is the probability that the actual state of the system is an element of the 
lattice of paths. The exact probability is derived from the minimalized lattice poly- 
nomial of paths by the method of inclusion-exclusion, also known as Poincare's Theorem. 
Dually the probability of failure, or system unreliability, is derived from the mini- 
malized lattice of cuts. Modularization and/or inversion can be helpful in keeping the 
probability function reasonably small in size. Computer software is available both to 
generate the probability polynomial and to calculate either the reliability of unrelia- 
bility. Approximations can be used based upon simplifying assumptions which delete low 
probability terms from the function. Conservative error bounds are obtainable for some 
models. The approximation techniques include serializing methods ("single-point fail- 
ures" and "parts count"), very large system approximations for fault-tree applications 
and the Esary-Proschan bounds. 


DISCUSSION 

System reliability analysis is closely related to Boolean logic. From George 
Boole's classic 1854 book, The Laws of Thought [6], it can be seen that the kinds of 
problems Boole was trying to solve 

... to make that method itself the basis of a general method for the 
application of a general method for the application of the mathematical 
doctrine of Probabilities . . . [6, p. 1] 

... it is always possible ... to express the event whose probability 
is sought as a logical function of the events whose probabilities 
are given ... [6, p. 15] 

is essentially the same problem that a system reliability analyst tries to solve. In 
this paper we interpret the objective to be that of logically symbolizing the success 
or failure of a system as a function of success or failure for various combinations of 
its components, and evaluating the probability of success, that is, the system relia- 
bility, as a function of the component reliabilities. 
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Boole made some errors. His explanation of "inversion" involved a clumsy use of 
series expansions and divisions by logical zero, in contrast to the more elegant 
de Morgan's theorems. Also he did not exhibit a deep understanding of "duality". Sub- 
sequent work by others based on Boole's foundations, led to many significant develop- 
ments in mathematics, symbolic logic and philosophy, including set theory, particularly 
lattice theory and the theory of partially ordered sets. Boolean algebra and minimal- 
ization by Quine [24, 25, 26]. The contributions of Boolean logic to applied disci- 
plines such as computer science and electrical engineering, particularly through the 
pioneering work of Shannon [30], are well known and recognized. 

Logically based system reliability analysis appears Lo have been initiated by 
von Neumann in connection with his search for ways of explaining how large digital 
computers having many thousands of unreliable components such as vacuum tubes could 
operate reliably. In a paper published in 1956 based on lectures given in 1952 at Cal 
Tech, von Neumann [35] showed that by the use of redundancy, it is possible to build 
and maintain a complex system having a greater reliability than any of its components. 

Moore and Shannon [20] used essentially parallel-series circuits consisting ex- 
clusively of idealized identical relays all with identical reliabilities’ to develop 
functions relating the reliability of a system R to that of the components. Bounds on 
this relationship were developed showing at what point R could be greater that the 
components. Mine [ 1 9 j generalized this work further by introducing Boolean and set- 
theoretic. notation, and employed linear graph theory to find the functional relation- 
ship between system and component reliability. He also developed bounds and conditions 
under which arbitrarily high R could be obtained. 

Birnbaum, Esary and Saunders [5] extended the Moore-Shannon approach to the general 
class of "coherent" systems that have certain logical consistency properties. That 
paper introduced terminology which has been widely used, such as "minimal paths", 
"minimal cuts" and "essential (relevant) components". Esary and Proschan [10, 11] ob- 
tained approximations for the reliability of coherent systems, including upper bounds 
based on the minimal paths and lower bounds based on the minimal cuts. The theory of 
coherent systems based on this general approach is discussed by Barlow and Proschan 
[3, pp. 1-51], and a review of the literature, with special reference to the contri- 
butions of Z. W. Birnbaum and his students and colleagues, is given by Saunders [28]. 

The research reported upon in the body of this paper deals primarily with general- 
ized approaches that apply not. only for a complex system of any configuration, but also 
if every component is different. Several large scale computer programs of this type 
were prepared in the early 1960's in connection with the U.S. space program. Some of 
these programs employed Monte Carlo. As a general rule, very little documentation is 
available as to what theory was employed or what was done. 

A FORTRAN software package with the acronym SCOPE (System for Computing Operation- 
al Probability Equations) was developed about 1965 at the Rockwell International 
Corporation. SCOPE provides an exact system reliability function of the component 
reliabilities for a complex system of any configuration but of limited size, based on 
either the minimal paths or else the minimal cuts. This function is derived by the 
method of .inclusion-exclusion, also known as Poincare’s Theorem. To obtain the system 
value, simply substitute the component values into this function. The SCOPE software 
is available through NASA [23], and the mathematical, theory is given in [15], including 
some comparisons to the Esary-Proschan bounds. 

An improved version of SCOPE was prepared in 1972 by Burris [7] at Oklahoma State 
University with the acronym MAPS (Method for the Analysis of the Probabilities of 
Systems). Instead of FORTRAN, it is programmed in PL/ 1 , which is better adapted to 
binary-digit manipulation. It also incorporates a modularity feature so that the 
system can optionally be processed as a complex configuration of independent modules, 
each module consisting of a complex configuration of independently failing elements. 
Consequently there is simultaneously both an increase in capacity and a saving in com- 
puter time, over the requirements of the parent SCOPE program. 

A further extension of MAPS is SPARCS (Simulation Program for Assessing the Relia- 
bilities of Complex Systems) programmed by Cooley [8] at Oklahoma State University 
under Air Force Contract F33615-74-C-4072. This uses Monte Carlo combined with 
Bayesian techniques to assess (provide a schedule of confidence levels for all values 
of R between 0 and 1) both R and the MTBF (mean time between failures) of a complex 
system of any configuration consisting exclusively of pass/fail and/or time-to-failure 
components, or any mixture of these, from failure-history data. Both MAPS and SPARCS 
are. described in [16]. A more efficient version of SPARCS, called SPARCS-II has been 


prepared by Lee [14]. 


A related and parallel effort to SCOPE and its daughter programs (or SPARES and 
its parents) is the development of the fault-tree methodology. Initially designed for 
aerospace applications at Bell Labs [4] in 1961, and subsequently also at Boeing Air- 
plane Company, as an aid to engineers in analyzing sequences of events leading to 
system failure, it has recently been used extensively for reactor safety studies by 
the Atomic Energy Commission and its successor, the Nuclear Regulatory Commission. A 
recent but historic document reporting the results of extensive applications of the 
methodology on the "Rasmussen Report WASH-1400" [31], particularly Appendix II, "lault- 
Trees". Descriptions of the methodology are to be found in Mearns [18], Haasl [12], 
Eagle [9], Schroder [29], Barlow and Lambert [3], Vesely [33, 34, 35 j , and Barlow and 
Proschan [3, pp . 264-266]. 

A fault tree is a Boolean-equivalent diagrammatic representation of all the ways 
of failing a complex system through combinations of failures and repairs of one or more 
components. It is derived from block diagrams, schematics, and/or blueprints, and 
logical analysis of the interrelationships of the elements. "Minimal cut sets" are 
obtained from the tree. By substituting the component values, an estimate of R, the 
system availability or the failure rate are obtained. A rather substantial, library of 
fault-tree methodology computer programs is available, including programs which build 
the trees, find the cut sets, or perform numerical evaluations. Vesely made some major 
contributions with the development of the PREP and KTTT codes [32, 33] , Salem, 
Apostolakis and Okrcnt. [27, pp. 34-45J and Worrell and Burdick [36] give reviews of 
the available software. 

In general , the fault-tree methodology incorporates the same Boolean and proba- 
blistic. theory that SPARCS and its parent programs do. However, because the method- 
ology is generally applied to large systems having only low failure-rate components, 
various types of "rare event approximations" are employed, to save computer time. 

These include both Monte Carlo and deterministic selections of components and cut sets 
according to importance, deleting higher order Intersection terms in the probability 
equation, and basing calculations on failure rates rather than probabilities. "Im- 
portance" measures are discussed by Nagel [21], Nagel and Schroder [22], Lambert [13] 
and Mazumdar [17] in [1], and Barlow and Proschan [3, pp. 26-29]. 

This article is a review of the state of the art of evaluating R for a complex 
system as a function of the reliabilities or failure rates of the elements. It devel- 
ops the common Boolean theoretical structure which underlies all the different 
methodologies, and shows some of the similarities of and the differences between exact 
methods and approximations. There are two parts: Part 1, on "Logical Formulation" 
and Part 2 on "Probability Calculations". 

Part 1 of the paper gives the logical, formulation, starting from set-theoretic 
first principles, from the viewpoint of lattice theory. We describe the universal set 
U of system states and power set & of lattices or events which are subsets of U. 

Events such as system success or system failure are collections of success states, 
called "paths", or else failure states, called "cuts", described by lattice Boolean 
polynomials. The terms of the minimalized form of the "success" polynomial, are the 
"minimal paths" and for the "failure" polynomial the "minimal cuts" where each term 
denotes a sublattice. 

Part 2 describes how to calculate the system reliability. First a probability 
function, a numerical-valued function of the component probabilities, is derived from 
the lattice polynomial by the method of inclusion-exclusion. Then the component 
probabilities are substituted into this function. Part 2 discusses exact methods, 
error bounds, approximations, serializing methods and the fault-tree methodology. 
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WHAT ARE QUALITY COSTS 


Joan H. Alliger* Product Control Engineer 
Eastman Kodak Company 
Rochester* New York 14650 


ABSTRACT 

In the area of quality control* the most efficient route 
to meet industries' challenges of improving product quality and 
lowering quality costs is to make the product (or provide the 
service) correctly the first time. Departures from this route 
incur unnecessary expense. 

With the proper tools for measuring quality cost data and 
a program for presenting quality cost information* trends can be 
established as guides for management in pursuing these challenges. 

The purpose of this presentation is to introduce a system to 
organize certain quality related costs enabling management to 
effectively measure and optimize quality costs as well as product 
quality. 

The system is described in detail in the ASQC publication* 
"Quality Costs - What and How", prepared by the Quality Cost 
Technical Committee, ASQC. 
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THE PHILOSOPHY AND USEFULNESS OF QUALITY COSTS 


William N. Moore, Manager Manufacturing Operations 
Westinghouse Electric Corporation 

Process Equipment Department, Sykesville, Md. 21784 


DEVELOPMENT OF THE GUIDE 

The "Guide for Reducing Quality Costs" was developed following 
a realization by members of the ASQC Quality Cost Committee that 
very little guidance existed on what to do with quality costs once 
they were defined, categorized and collected as described in "Qual- 
ity Costs" - What and How". Several of the members of the committee 
had experience in setting up and implementing successful quality 
cost reduction programs. Several of the companies represented had 
internal guides and literature on the subject. It was felt that 
quality management and general management would benefit from a 
how-to book dealing with what uses could be made of quality costs 
and the structure of programs to reduce the costs. The "Guide for 
Reducing Quality Costs" is an attempt to tell management: 

. How to analyze their quality programs using quality costs. 

. How to structure programs for reducing costs. 

. How to find areas needing improvement. 

. How to reduce failure costs. 

. How to reduce appraisal costs. 

. How to prevent quality costs. 

. How to measure improvement. 

The guide represents many hours of work on the part of those 
people listed in the front of the guide. My presentation in this 
session will closely follow the content of the first five sections 
of the guide: 

1. Purpose and Quality Definitions 

2. The Quality Cost Improvement Philosophy 

3. Usefulness of Quality Costs 

4. The Quality System 

5. Finding the Problem Areas 

Following sections of the paper summarize the content of each 
section of the guide, but do not repeat the content or contain 
detailed explanations. The reader is referred to the guide for 
such detail. 


PURPOSE 

The purpose of the publication is to provide guidance to 
general management and quality program management to enable them 
to structure and manage programs for Quality Cost Reduction. The 
guide provides the procedures for taking total quality costs and 
using them for identifying problem areas and for reducing these 
costs. Quality costs covered include those in the design, manu- 
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facture, inspection, test, and product service phases. Improve- 
ment programs must encompass all phases of product life, from 
design through use by the customer. 

THE QUALITY COST IMPROVEMENT PHILOSOPHY 

The guide is about a quality program not confined to the con- 
trol of quality in the manufacturing stage. Most people recognize 
that product quality is determined by many factors outside this 
stage, but many quality programs do not concern themselves with these 
factors. In some cases, quality program efforts have been attempts 
at not allowing things to get any worse (control) instead of striv- 
ing to get better (improvement). As a result, things have gotten 
worse in many places, simply because controls are not 100% effective 
and can never be. Improving quality is much like improving product 
costs. It is everybody's job and everybody is for the idea, but, 
until there is a management commitment to improve and a formal 
program for Forcing Tmprovemeht, it just doesn^t happen . 

The guide describes what each function must do to satisfy the 
customer's needs and reduce your quality costs. Also described are 
ways to prevent the production of defectives through involvement of 
people in Marketing, Design, Purchasing, Accounting, Manufacturing, 
and Quality Assurance. It describes ways to find problems and 
correct their causes. It tells you how to use the costs associated 
with quality and how to reduce those costs. 

Quality improvement results in cost improvement. Designing 
and building a product right the first time always costs less. 

Solving problems with existing products by finding their causes and 
eliminating them results in measurable savings. To cash in on these 
savings requires that the quality performance of the past be improv- 
ed and the guide describes ways to do that. 

The chart in the guide shows how quality cost analysis bridges 
the gap between the elements of a prevention oriented quality pro- 
gram and the means used by general management to measure performance 
- the profit and loss statement. 

USEFULNESS OF QUALITY COSTS 


Quality costs are useful for strategic planning and program- 
ing quality improvement. 

STRATEGIC PLANNING 


Strategic quality program planning is vital to the continuing 
profitability of many segments of American Industry. The pressures 
for safer, cleaner, and more reliable products are becoming stronger 
each year. We must find ways to meet these increasing demands and 
still remain competitive. The key to doing this is to improve 
quality using the methods described in this guide, and reduce costs 
as a result. 

To improve quality and reduce quality costs there has to be a 
trigger for making changes in the status quo. The firm's Strategic 
Plan is an ideal device to force changes; and the inclusion of 
quality and quality cost improvement plans in the overall strategic 
plan is recommended. This gives the quality and quality cost situa- 
tion the management visibility too often lacking. Because quality 
and quality cost improvements are set forth as business objectives 
(along with the more conventional business objectives) against 
which management performance is evaluated, there is effective moti- 
vation for action. 



Relationship Between Quality Costs and Strategic Planning 


Quality costs for a profit center are made up of costs in- 
curred in several activities. The chart in the guide shows the 
buildup of costs from all functional departments into an overall 
quality cost analysis for the entire profit center. 

Quality costs are incurred by all major functions in an or- 
ganization, so problem areas can exist anywhere. Careful analysis 
must be done to find the most costly problems and programs must be 
developed to attack them. Many times a strategic program is needed. 
When this need exists, a strategic quality program should be de- 
veloped using inputs from all functions and it should become a part 
of the profit center's overall strategic program. 

The Planning Process 

Strategic planning should be done in a step by step process. 

The basis steps are: 

. Review past performance and present position 
. Appraise the environment 
. Set objectives 
. Select a strategy 
. Implement the strategic program 
. Report and evaluate the plan 

The final step in the cycle provides needed inputs for the first 
step so planning becomes a continuous process. Following are general 
considerations for accomplishing each step in the cycle. 

Review Past Performance and Present Position - A thorough review 
provides a realistic assessment of past performance, current condi- 
tions, and future potential. 

Appraise the Environment - There are numerous environmental factors 
which may significantly interact with the quality programs. Typical 
examples are: 

. The activities of other departments. 

. Changes in customer demands. 

. New safety and liability regulations. 

. Actions of competitors. 

Set Objectives - From the knowledge and understanding achieved in 
the status review and the environmental information, the strengths 
and weaknesses of the quality program should be known. So, speci- 
fic objectives with target completion dates can be established. 

Select a Strategy - Once clear objectives have been established, a 
definite strategy should be formulated and clearly stated. 

Implement the Strategic Program - Planning for the implementation 
of the strategy is the most important step of the planning process. 
All prior effort leads up to the implementation or action step. 

There are many cases of "that was a sound strategy but nothing 
happened as a result". This situation is due to a failure to follow 
up the strategy with appropriate action programs. 



Report and Evaluate the Plan - The final step in the planning 
cycle is one of integrating the strategic quality plan into the 
total strategic plan and evaluating the costs and benefits of the 
plan. 


Through better planning, quality performance can be improved. 
Continued achievement of good performance can identify the quality 
program as a key contributor to the success and assure that Quality 
will play an enlarged role in future plans and activities. 

PROGRAMMING IMPROVEMENT 


The Strategic Quality Plan describes a management commitment 
to quality and quality cost improvement. The quality cost data 
indicate the areas that are candidates for improvement . When the 
highest cost areas are analyzed in greater detail many improvement 
projects become apparent. For example, high warranty costs are a 
trigger to rank customer failure problems for detailed investiga- 
tion, with the aim of looking into product design, process control, 
or inspection planning for the cures to the highest cost problems. 
Regardless of what the high quality cost category may be, the mere 
act of identifying it should lead to actions to reduce it. 

To effectively program quality improvement efforts, it is 
necessary to: 

. Recognize and organize quality related costs to gain 
knowledge of magnitude, contributing elements and 
trends . 

. Analyze quality performance, identify major problem 
areas and measure product line and/or manufacturing 
section performance. 

. Implement effective corrective action and cost improve- 
ment programs. 

. Evaluate effect of action to assure intended results. 

. Program activities for maximum dollar pay off and maxi- 
mum effective manpower utilization. 

, Budget quality work to meet objectives. 

THE QUALITY SYSTEM 

Perhaps the most important result of the collection, analysis, 
and use of quality costs is the exposure given to the total quality 
system as it really exists in the organization. The collection of 
costs forces definition of all activities contributing to the qual- 
ity of the product. Analysis of quality cost data forces evaluation 
of the effectiveness of the contribution of each activity, the re- 
lationships among the many activities, and the all-important commun- 
ications links that tie activities together. 

Each organization will define its quality system differently , 
but the overriding requirement is that whatever definition is used 
must be comprehensive. That is, it must include all efforts that 
affect product quality , wherever the efforts are accomplished in 
the; organization. 

If there is a weakness in the quality system, it is usually a 
deficiency in the integration of the elements and their sub-elements 
into a working whole. Almost every manager can look at his system 
elements and convince himself that his organization has something 
going in each area - perhaps he even has a shelf full of procedures 
manuals to prove it. However, much of what is actually going on in 
the quality system might be superficial, or might not be well inte- 
grated into the operations and traditions of the total organization. 



The concept of quality costs is a potent tool for management 
precisely because it can be used to force the integration of all 
the separate quality activities into the mainstream of the product 
cycle; that is, into a total quality system. It forces the entire 
organization to examine each cost element (and each quality-related 
activity) in the context of the total quality cost (and total 
quality system) . 


FINDING THE PROBLEM AREAS 


When quality costs are displayed to managers who have not been 
exposed to the concept, the initial question is likely to be "how 
much should they be?" or "how does this compare with other organi- 
zations or products?" Unfortunately, it is not practical to estab- 
lish any meaningful absolute standards for such cost comparisons. 

A quality cost system should be "tailored" to a particular company's 
needs, so as to perceive trends of significance and furnish objec- 
tive evidence for management decisions as to where assurance efforts 
should be placed for optimum return. The search for "industry 
guidelines" or other standards of comparison, while natural, is 
quite dangerous, since it leads to quality costs emphasis for 
"score-carding" instead of utilization as a management tool for 
improving the status-quo. 

The futility of establishing meaningful absolute quality cost 
guidelines is more apparent if you just consider: 

1. Inherent Key variations in how companies interpret 
and capture quality cost data; 

2. Critical differences in product complexity, process 
methods and stability, production volume, market 
characteristics, management needs and objectives, 
customer reactions, etc; 

3. The awkwardness or inappropriateness for many 
companies of the most prevalent form of quality 
cost measure (% of net sales billed) , considering 
effect of time differences between time of sales 
billing and incurrence of actual quality costs. 

This last factor is particularly important for periods involv- 
ing an expanding or contracting product volume or mix, unstable 
market pricing, shifting sales/leasing revenue ratios, or changing 
competitive performance criteria. Accordingly, it is much more 
productive to abandon efforts to compare your quality cost measure- 
ments with other companies, in favor of meaningful analysis of the 
problem areas contributing most significantly to your quality costs, 
so that suitable corrective actions can be initiated. 

Analysis techniques for quality costs are as varied as those 
used for any other quality problems in industry. They range from 
simple charting techniques to complicated mathematical models of 
the program. The most common techniques are: 

1. Trend Analysis 

2. Pareto Analysis 

A. By Element Group 

B. By Department 

C. By Product 

D. Other Groupings 

Trend Analysis is simply comparing present cost levels to past 
levels. It is suggested that costs be collected for at least one 
year before attempting to draw conclusions or plan action programs. 
The data from this one year (minimum) period should be plotted in 
several ways. 



The Pareto Analysis technique involves listing the factors 
that contribute to the problem and ranking them according to the 
magnitude of their contribution. In most situations, a relatively 
small number of causes or sources will contribute a relatively large 
percentage of the total costs. To produce the greatest improvement, 
effort should be spent on reducing costs coming from the largest 
contributors . 
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INTRODUCTION 

The costs of appraisal sometimes approach half of the total quality 
costs. Although most quality cost improvement programs properly concentrate 
on reducing failure costs first, programs for appraisal cost improvement can 
also have a significant impact. We will discuss several techniques for 
improving these costs. 

Inspection and Test Planning 
Equipment and Methods Improvement 
. Statistical Quality Control 
. Appraisal Accuracy Studies 
. Decision Analysis 
. Work Sampling 


INSPECTION AND TEST PLANNING 

Getting the most out of the available appraisal resources requires careful 
planning. Determining where control points should be and the amount of inspec- 
tion and test to do should be the job of professionals - not left to the judg- 
ment of the individual doing the inspection or test job. 

In-process controls are a vital part of a prevention oriented quality 
system. They provide a powerful means for reducing the incidence of defective 
finished product and for reducing quality costs. In addition, an effective 
in-process inspection system often makes it possible to reduce the amount of 
final inspection required. In-process inspection control involves inspections 
or verifications performed at significant stages of the manufacturing process. 
If defective parts or subassemblies are being produced, the trouble can be 
detected early and corrected before it affects the quality of the finished 
product. The system must be efficiently designed so that every inspection 
will serve an essential purpose. 


The requirements that the finished product has to meet, along with 
finished product inspection and test specifications, should be thoroughly 
reviewed during the development of the in-process inspections and controls. 
This review provides the Quality Engineer with the information needed to 
determine the type and degree of in-process inspection required at various 
stages of the manufacturing process or assembly operations. It also helps 
him select the kinds of in-process controls that will prevent the manufacture 
of defective product and yield the best economic return. A periodic review 
of the planned inspection and tests should be made to assure that the levels 
remain economical in light of quality history. We will briefly review five 
types of inspection and test controls and the advantages and disadvantages of 
each: 


. Operator Inspection 
. 100% In-line Inspection 

First Piece Inspection 
. Patrol Inspection 

In-Process Acceptance Inspection 
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Operator Inspection 

In-process controls can be enhanced by requiring an operator, stationed 
at a machine or at a processing station, to inspect his own work. The 
operator must be provided with the proper gages and be instructed in their 
proper use. He should be trained to recognize when an item is unacceptable 
in appearance. Enough time should be provided in his work standards to allow 
him to perform the inspection with reasonable care. 

These are the advantages of operator inspection: 

. The operator usually handles every piece coming off the line. 

. He is thoroughly familiar with the item he is making. 

. He is in a position to spot defects quickly and call for help to 
correct problems as soon as they appear. 

One caution is that special care may be necessary for the operator to keep 
the records normally required for an effective inspection procedure. 

100% In-Line Inspection 

Inspection or testing may be carried out on a 100* basis at designated 
points in the manufacturing line. This type of inspection or testing is 
normally performed by inspection or test personnel in the Manufacturing Depart- 
ment. Its purpose is to screen out items that either do not conform to quality 
workmanship standards or are not likely to pass the finished product inspection. 

The following are some of the advantages and disadvantages of 100% in-line 
inspection. 

Advantages 

It saves the cost of further processing of a product that is likely to 
fail final inspection and test. 

It provides data on quality performance that can be used to take 
corrective action. 

Disadvantages 

" The Tn-1 ine inspection function may become a routine step in the manu- 
facturing line and because the rejects are being screened out, there 
may be less emphasis on the prevention of defects. 

It tends to duplicate inspection and increase inspection costs. 

It is not 100% effective because performing 100% inspection does not 
guarantee that all of the defective items will be detected. 

First-Piece Inspection 

In first-piece inspection, several pieces at the beginning of every new 
run are inspected to determine whether the set-up has been properly made and 
whether the tooling is adequate. The sample should provide a complete check of 
the machine or operation set-up. If the machine has nine spindles, for instance, 
samples should be taken from each spindle. Usually, the first five pieces 
produced by the new set-up constitute a large enough sample. 

The advantage of first-piece inspection is that since the items turned 
out by a process or operation are evaluated at the beginning of the run, &ny 
necessary correction can be made before the run is started. 

Patrol Inspection 

The inspector patrols the operations at periodic intervals and inspects 
the items being produced. Since inspection is performed concurrently with the 
operation, patrol inspection can provide faster response than inspection after 
the item has been completed. 

It is advantageous to use patrol inspection under the following conditions: 


. When a process turns out a high percentage of defective products and 
requires frequent inspection. 

When a process is erratic and the operator is unable to do a thorough 
job of inspection. 

. When there is a need to collect special detailed data on the performance 
of the process. 

. When an audit of the process is required. 

In-Process Acceptance Inspection 

This is the classic type of inspection. All the items made at an operation 
in a given period of time are inspected together as a lot. They must be 
inspected before they are approved for release to the next operation. 

In-process acceptance inspection provides several advantages: 

It makes it possible to control the quality level at each successive 
stage of the manufacturing process. 

It provides data to use in preparing performance reports to help 
pinpoint problem areas. 

On the other hand, it also has some limitations: 

It does not prevent defects since the inspection is performed after 
the items have been completed. 

It delays the movement of parts from one operation to the next. 

. It is not easily applied to continuous processes. 

Of the five types of inspection, none is completely effective by itself. 

An efficiently designed in-process inspection system requires several of them 
in combination. How they can best be combined to serve particular needs 
depends on an evaluation of the following factors: 

. The cost of each type of inspection. 

. The type of manpower each type requires. 

. The history of quality performance. Has the process been in control 
in the past? 

. The type of process. Is it continuous, or can the items produced be 
collected and inspected in batches? 

. The stability and the capability of the process. 

. The nature of the product characteristics being controlled. Are the 
characteristics critical or minor? 

IMPROVING EQUIPMENT AND METHODS 

Many of the most profitable areas for savings of inspection and test costs 
lie in improved equipment and methods used to do the job. Since inspection 
and test are not usually measured and controlled to the extent production jobs 
are, they are not usually too efficient. Improvements can often be made by: 

1. Providing equipment which can perform inspection and test tasks faster 
or without operators. 

2. Building inspection or test devices into production equipment. 

3. Designing improved record and reporting systems which require less 
time and effort. 

4. Applying industrial engineering techniques to improve inspection and 
test station layouts and methods. 

STATISTICAL QUALITY CONTROL 

Powerful tools that can be used to help achieve ^-process control are 
capability studies, control charts and sampling inspection. 

A capability study shows whether a machine or process is inherently capable 
of turning out items that conform to specification. The results of the study 
may even indicate that it is possible to reduce the amount of inspection without 
adversely affecting quality. On the other hand, a capability study may reveal 
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that a certain percentage of the items will always exceed the tolerances of 
the specification. In this case, it may be necessary either to relax the 
tolerances or to acquire a machine that is capable of meeting them. 

Control charts are another excellent tool for increasing the efficiency 
of the in-process control techniques. In any series of measurements, there is 
variation. Sometimes the variation is only the natural outcome of "constant 
causes" inherent in the process. In other situations, there is also variation 
due to what are called "assignable causes". In the first case, the variation 
is normal and the process should be left alone. In the second, the variation 
indicates that something has gone wrong with the process and action should be 
taken to correct it. The problem is to decide which type of variation is present. 

Acceptance sampling techniques provide a means of measuring and controlling 
quality without the necessity of checking all the units produced. By using a 
sampling plan, it is sometimes possible to significantly reduce the costs 
associated with appraisal while still maintaining adequate control. 

ACCURACY STUDIES 

There are failure costs associated with incorrect quality decisions on the 
part of inspectors, testers, or operators. These can be in the following areas: 

1. Falsely rejecting acceptable material. 

2. Falsely accepting rejectable material. 

There are many plans which rate appraisal personnel in relation to these 
two errors. Perhaps the easiest to apply is one involving accuracy as a percent 
of defects correctly identified. This involves submitting a known number of 
good and bad units to an individual and rating his ability to correctly separate 
the units. The number of incorrect decisions is then multiplied by the cost of 
each wrong decision and an extension is made showing the cost implications. 

DECISION ANALYSIS 

In the early manufacturing of a new product (and in spite of good quality 
planning), the need for adjustment of measurement and test controls is generally 
revealed. This creates a need to analyze the effectiveness of decisions made 
on components, sub-assemblies and final product in terms of the earliest possible 
detection of defects. A technique called decision analysis is sometimes helpful 
in such a determination. This involves an analysis of accept-reject decisions 
of inspectors and testers, and identifies the point in the process where such 
decisions are made. Further, the model or part on which such decisions are made 
is also specified. 

Summaries of such results frequently show trends for individual inspectors, 
especially where inspection planning, visual standards or training is less than 
adequate. These trends show up in terms of two inspectors servicing the same 
area and rejecting significantly different amounts of material. Also, decision 
times (i.e., the time required to make an accept-reject decision) may be 
significantly different for two inspectors in the same area. 

Obviously, the planning engineer must address to these differences and 
provide improved control (and therefore improved costs) by more effectively 
utilizing the appraisal personnel. 

WORK SAMPLING 

The technique of work sampling consists of sampling work elements of an 
individual or group and using probabilistic theory to estimate the total time 
spent on a given activity. When applied to appraisal personnel, who very often 
do not have repetitive work elements, it can be used to more effectively 
structure the work routines. For example, if work sampling determines that 10 % 
of an inspector's time is spent walking from one end of the department to another, 
then obviously some change in his geographic assignment or work station could 
be made to minimize such a cost. 


Generally, work sampling has been found to be a better tool for measuring 
indirect labor rather than direct labor, inasmuch as repetitive work elements 
can be studied by either time study or predetermined time standard systems. 

REDUCING APPRAISAL COSTS 

In summary, we can (and should) ask some pertinent questions to determine 
the effectiveness of our appraisal efforts: 

1. Are inspection points located to maximize the return on dollars spent 
for inspection? 

2. Are inspection stations and methods engineered for the most efficient 
work accomplishment? 

3. Could inspection and test operations be economically automated by using 
special purpose instrumentation, or tape or computer controlled equip- 
ment? 

4. Could inspection and test record and data reporting functions be more 
efficiently performed using the computer or other modern data handling 
devices? 

5. Is it possible to control processes sufficiently to prevent production 
of defectives and eliminate product inspection? 

6. Could statistical Quality Control techniques be profitably used? 

7. Could tests now being performed by outside laboratories be performed 
at less cost in-house or vice versa? 

8. Are some tasks now being performed by highly paid inspectors or testers 
which could be performed by lower classification employees? 


REFERENCES 

(1) "Guideline for Reducing Quality Costs", American Society for Quality 
Control, Milwaukee, Wisconsin, 1977 
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REDUCING FAILURE COSTS AND MEASURING IMPROVEMENT 


William O. Winchell, Project Manager 
Product Assurance Department 
Environmental Activities Staff 
General Motors Technical Center 
Warren, Michigan 48090 


Today I would like to discuss with you two topics — reducing failure costs and measuring 
improvement — from the ASQC publication "Guide for Reducing Quality Costs." Reducing 
failure costs is a tough thing to do and I am sure you will agree that once it is reduced, it is 
even tougher to maintain the new level. Eliminating failures is something like an old leaky 
tube of toothpaste. You can fix the leak and force the toothpaste in the right direction — 
through the nozzle; but not for long — soon it leaks in another spot. But there is a reason 
for this. It is because in most cases the real cause of the problem has not been identified. 
Without knowing what the real problem is, we surely can't fix it. 

So often the quality manager sees his job only as informing management of symptoms of 
problems such as high rejects and excessive returns from the field. His fix, considering his 
perceived scope of activity, is to put more inspection on to keep the bad parts from escaping 
until that day comes when the problem is corrected. But without effective corrective action 
that day never comes. Management looks to the quality organization for more than this and 
rightly so. The quality organization must not only report problem symptoms, it must involve 
itself in the mainstream of defining the real problem and proposing the best possible solution. 
The quality organization must develop an effective corrective action system that gets at the 
roof cause of problems in order to permanently reduce failure costs. 

The ASQC publication "Guide for Reducing Quality Costs" recommends four key ways of 
reducing the cost of failures. These are to: 

Communicate the problem area to all affected persons. 

Create a desire in others to mutually solve the problem. 

Provide leadership in the planning and carrying out of a logical 
investigation by those in the company best able to resolve the problem. 

Follow up to insure that the problem was permanently fixed. 

Quality performance reports are a good fool for communicating problem areas to those 
that can solve the problem and to make top management aware of the progress. However, 
they must be understandable, clear and to the point. They should summarize information and 
not overwhelm the user with needless detail, while pointing out and emphasizing the significant 
trends. Certainly, the reports should be designed to meet the needs of those using it for 
faking corrective action, as well as for those fracking the progress. Obviously, there is no 
one best format; the reports must be individually designed to meet the needs of your organiza- 
tion. They must be user oriented. If they are not they will be useless documents on the 
desks of those people that the quality manager needs most -those that can find and fix the 
problem. 

Creating a desire in others to mutually solve a problem is critical for most qualify 
managers. If is directly reflected in the quality manager's ability to solve the chronic qualify 
problems existing in his company. This is because the causes of the chronic problems are not 
normally the ones that the quality manager can correct by himself. A chronic problem may be 
caused by bad designs, improper fooling, improper manufacturing sequences or many other 
things which, of course, must be corrected by someone else. To get someone else to do 
something about the problems is a selling job, to a large extent. For the qualify manager this 
involves selling himself, his corrective action program and, most importantly, the fact that the 
problems are serious enough to justify spending the time and money necessary to solve them. 
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If Is important to remember that in order to effectively sell your programs, good justifi- 
cation must be the comer stone. In the early stage of investigation the cost of the problem, 
the amount of effort needed for solution and the tangible and intangible benefits should be 
estimated as best you can and presented in your corrective action reports. Adequate justifi- 
cation will gain support from your management when the problem is stated in terms they under- 
stand. For example, the engineering department can often be stimulated to correct a design 
problem when it is shown that it affects product reliability and that a specific amount of 
excessive warranty cost is being generated. The production supervisor must be convinced that 
a solution to a rework problem will help him in his daily battle with the efficiency of his 
operation. Above all, don't give up if your best selling job falls on deaf ears the first time 
around. If you are still convinced that the solution to the problem will contribute positively 
to your company, look for ways around the road blocks. Remember, the qualify manager must 
provide the leadership for failure reduction. 

Of course, a key to your failure reduction program is a systematic corrective action 
system through which logical investigations and resolutions of problems can be processed. The 
ASQC publication "Guide for Reducing Quality Costs" suggests several forms that can be used 
as a framework around which your system can be designed. Experience has shown that certain 
aspects need to be determined and documented early to insure the effective operation of your 
system. They are: 

A step-by-step plan for investigating and solving the problem. Each 
step should have the responsible individual listed along with a target 
date for completion. It is vital that the plan be kept up to date, 
reflecting changing conditions, and that periodic status be reported. 

- Both tangible and intangible benefits expected to be achieved in terms 
management can understand. 

The projected cost to achieve the desired results. 

By having a plan, those involved will be confident that progress will be made and that 
the benefits at the end of the road outweigh the costs that will be incurred along the way. 

In addition, priorities can be systematically assigned to problems so that those having the 
greatest benefit to your company will be attacked first, while those having smaller payoffs 
wait on the back burner. 

Following up to insure corrective permanent action boils down to a periodic measurement 
of the results. The initial measurement will determine if the plan is achieving the projected 
results at the estimated cost. If if did not, additional Investigations to obtain the needed 
results should be launched. Periodic remeasurements, either as part of your formal quality 
reporting system or by special studies, will help insure that the gains are permanent. Holding 
the improved quality levels is vital and often overlooked. It is common to see improvement 
trends reverse themselves and head toward former levels. By proper reevaluation, these down- 
ward trends can be caught early and corrected before they become serious. 

In summary, a program to reduce failure costs has to consider: 

- Communication of problem areas to all those affected. 

- Creating a desire in others to mutually solve the problem. 

Providing leadership in the planning and carrying out a logical 
investigation of the problem with those best able to resolve the 
problem. 

Following up to insure that the problem was permanently fixed. 

Now let us discuss our next topic, which is included in the ASQC publication "Guide 
for Reducing Quality Costs, " how can we measure improvements in a company's quality system? 
Being a quality cost session, you might think that we would stress that quality cost is the 
best way of measuring improvements, but this is not always true. Only sometimes can quality 
cost provide the answer and, when it is used. It must be applied with a great deal of care 
and discretion. The ASQC publication "Quality Costs - What and How" is a good reference 


to help you stay clear of the pitfalls inherent in comparing costs. 

For example, don't be misled in comparing percentages. One might assume that a failure 
cost decrease from 70% of total quality cost to 60% is a big improvement. But if total 
quality cost, on a unit basis, rose between the two periods from $2. 00/unit to $2.5C/unitdue 
to added inspection, failure cost on a unit basis actually increased. Indeed this company is 
still in trouble. Quality costs are usually good for determining the magnitude of in-plant 
improvements that require the measurement of inspection, scrap or rework costs. But again, be 
careful. Don't make conclusions on records alone — use personal observations too. Take a 
walk to the production line and see if things are what the records indicate. So often an 
innocent mistake in recording information or a major shift in the product mix to something 
easier to build will cause results on paper to misrepresent what is really happening. There is 
really no substitute for a first hand look at the situation. 

Perhaps the greatest pitfall to guard against in using quality costs is to measure the 
effect of product improvements relative to field failure or warranty. To illustrate this, let's 
suppose that we have a reliability problem that only shows up after six months in the 
customer's hand. Assume that we fixed that problem in January, 1978, Even if the product 
goes directly to the customer, it is July of that year before the effect on warranty cost is 
felt. Obviously, it is absurd to use warranty expense for the month of May to track the 
difference. This inherent lag makes it tough to apply quality costs to finding out if field 
performance Is better, especially if your boss is breathing down your neck for an answer. 
Something else must be used to obtain timely answers such as life cycle testing or possibly 
outgoing quality audits of completed products that incorporate durability provisions. Such 
tests can discover system or product discrepancies far more quickly and at less expense than 
would be associated with waiting for failures in the customer's hands. 

Other sources of data besides warranty failure costs that are good indicators of field 
performance, depending upon each individual situation, are: 

Field trouble reports originated by perhaps your field service engineers. 

These provide more detail regarding your quality problems allowing a 
more accurate comparison between present and past performance. 

Market research surveys that determine what the customer thinks about 
your product. This indeed may be an appropriate way of getting a 
handle on potential lost sales due to quality problems. 

Spare part sales that supplement warranty failure cost information may 
better identify product performance trends. 

Customer complaints, properly tabulated and summarized, are another 
means of tracking progress. However, caution must be exercised in 
that this data may not represent your normal customer — they usually 
are from the most irate of customers. 

Installation phase reporting of quality problems found in the erection 
and commissioning activities of large and complex projects certainly 
is a valuable source for tracking progress for these type of products. 

In summary, there are many ways besides quality cost to track improvements in a 
company's quality system. The choice of which method or methods to use depends upon your 
individual situation. One method that is inadequate for one situation may be the best in 
another. Most importantly look before you leap. Carefully look at all the alternatives before 
selecting those that you will use for your immediate needs. 
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DLA CONTRACTOR ASSESSMENT PROGRAM 


James C. Albini 

Deputy Director, Quality Assurance 
Defense Contract Administration Services Region, Cleveland 
Cleveland , Ohio 


INTRODUCTION 

In January 1975, the Defense Logistics Agency (DLA), at the request of the Depart- 
ment of Defense (DoD) Quality Assurance Council, undertook the task of developing a 
concept for cerifying defense contractors with superior quality history. After evalu- 
ating certification programs utilized by other government agencies, NATA countries and 
private industry, a plan was formulated which combined the best features of all the 
techniques considered as well as others developed by DLA. The basis for the plan is 
predicated on the DoD philosophy of quality assurance, which states that responsibility 
rests upon the supplier for the quality of their product and that utilization of Govern- 
ment Quality Assurance personnel must be prioritized based on the contractor's demon- 
strated control of product quality and the effectiveness of his quality system. The 
concept, covering most DoD procurements, was approved in April 1976 and steps were 
taken by DLA for a trial implementation at select contractor's plants starting in 
November 1976. 


INITIAL SELECTION OF CONTRACTORS 

In the selection of contractors for trial implementation, Phase I of the test pro- 
gram, DLA considered only prime contractors who: (1) met the requirements of military 
specification MIL-Q-9858A, which requires the contractor to establish an effective and 
economical quality program throughout all areas of contract performance; (2) had inspec- 
tion and acceptance at source; (3) met the rigid selection criteria; and (A) had a 
history of delivering superior quality products. After carefully evaluating the 
possible participants, the RCA plant at Morrestown, New Jersey and the Bendix plant at 
Towson, Maryland, were chosen as sites for the test. 

The senior executives of RCA and Bendix agreed to participate in the test program. 
Their commitment to the program was on a strictly voluntary basis and with the under- 
standing that no contractual changes would be made, no overall increases in costs would 
be incurred, the program could be terminated at any time by the contractor or the gov- 
ernment without prejudice, and the number of government personnel at the facility would 
be reduced to a minimum consistent with the quality of the products and the overall 
government quality assurance in-plant workload. This in no way infers that there would 
be an abrupt removal of Government personnel from a contractor facility. Rather, a 
Government Personnel Plan is developed for the facility and takes into consideration 
the manpower and skills requirements geared to the tasks to be accomplished. Thus, the 
reallocation of resources, if necessary, is gradual and then only to concentrate it 
more efficiently in other quality areas, similar to that which occurs as a result of 
normal workload expansions and contractions. 

GOVERNMENT QUALITY ASSURANCE PLAN 

The main thrust and heart of the program is the tailoring of a detailed Government 
Quality Assurance Plan to the facility, its programs and products. The plan provides 
for verifying the contractor's control of quality as well as an independent evaluation 
of the quality of products. It further provides for the maintenance of sufficient ob- 
jective quality evidence which will provide traceability and for the continuous tracking 
and use of quality data to determine effectiveness and adequacy of the quality program. 

The Government representative responsible for planning, developing, implementing 
the contractor Assessment Program (CAP) at a given contractor facility is identified 
as the Government Quality Assurance Manager (QAM) . This title was selected to empha- 
size his greater responsibilities, considerably broader and more demanding than those 
of the typical DLA Quality Assurance Representative. Cooperation between the Govern- 
ment and the contractor is stressed in developing a Government Quality Assurance Plan 
that will assure effective control in a plant. The monitoring and review functions 
performed by the contractor in accordance with specification MIL-Q-9858A, Quality 
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Program Requirements, are primary considerations. The plan describes quality tasks 
that are important to overall quality in the plant; the procedural aspects as well as 
the person or organization responsible for each of these tasks; the manner in which the 
contractor will perform his monitoring and review actions; and how the Government will 
verify those actions. The frequency of Government verification is determined by the 
extent of control exercised by the contractor. 

IMPLEMENTATION AND RESULTS OF PHASE I 

The trial implementation of the Program at RCA and Bendix began in November 1976. 
Comprehensive data was collected and analyzed biweekly, historical data was compared 
with test data. The overall analysis of Phase I was made in July 1977 by DLA and 
representatives of the military departments and found to be working satisfactorily; 
that the program was effective in meeting its objectives; and that it had the potential 
as a viable management tool for both government and industry. It clearly illustrated 
the benefit of the program in terms of the increased efficiency of contractor line 
inspectors and the lower percent of defective items found by the company's quality 
auditors during their subsequent examination of the product. The number and severity 
of defects found by the inspectors and auditors over a six-month period provided a 
good measure of product quality inasmuch as they are quality indicators that can be 
easily understood and analyzed. Results of Government product verification and evalu- 
ation of the contractor's quality procedures during the same period significantly re- 
flected the overall improvement in quality at both test facilities. 

Because, the major programs at the two test facilities were developmental in nature 
rather than production oriented, the decision was made to extend the test over a longer 
period of time and to include facilities with production contracts in order to determine 
the true worth of the program. Field usage, the ultimate yardstick by which product 
quality can be measured and user satisfaction determined, was not available during 
Phase I of the test because of the nature of the contracts at RCA and Bendix, but: will 
be under the expanded version of the program. The program was implemented on one. 
product line at Texas Instruments, Dallas, Texas, in May 1977 and analysis of the 
results disclosed that it is working well there. 

IMPLEMENTATION OF PHASE II 

Phase II of the test program was implemented in early 1978 and will extend into 
1980. It was determined at the onset that a select number of facilities with pro- 
duction contracts would be added to broaden the data base and make it possible to 
assess the quality and reliability of fielded material that might best be approached 
by reviewing available field user data. Such participation should contribute in an 
important way to the determination of user satisfaction and at the same time provide 
the opportunity for a shakedown and refinement of the procedures, if necessary, which 
will give both Government and industry the added confidence to proceed with the program. 

The Cleveland Pneumatic Company, Cleveland, Ohio, a company which produces landing 
gear for military aircraft and with a history of outstanding quality products, was the 
first production facility to be considered for the Phase II extended test. The program 
was implemented there on 1 March 1978 and results thus far indicate that it is pro- 
ceeding satisfactorily. Others will be brought into the program as rapidly as possible. 
It is expected that the military departments as well as the DLA Contract Administration 
Regions will recommend contractors for inclusion in the Program. Final selection will 
be made by the Contractor Assessment Board (CAB) which is composed of senior quality 
representatives of the military departments and DLA. Candidates will be selected on 
the basis of restrictive qualification criteria, primarily an established history of 
high quality performance. A survey to determine any adjustment in government manpower 
needs is conducted six months after implementation of the program in a facility, com- 
mensurate with the contractor's level of performance over that period of time. A semi- 
annual evaluation of each contractor's performance will also be made by the CAB with 
summaries of quality data furnished to government Program Managers every three months. 

A follow-up quality system review is conducted at all CAP facilities 12 months after 
implementation to determine the effect to the program on product quality. 

The CAP can be advantageous to both Government and industry. Participating con- 
tractors, through encouragement to improve the quality of their product, will be 
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rewarded with greater production efficiency. The program has promise for cost reduc- 
tions and improved efficiency in Government quality assurance. It is expected to play 
a significant role in the shaping of quality and reliability programs of the future, 
and has the potential and promise for the next major advance in procurement concepts. 
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FABRICATION INFORMATION SYSTEM 


IRVING R. BLUMENTHAL, HEAD 
FABRICATION/PROCESS PRODUCT ASSURANCE 
HUGHES AIRCRAFT COMPANY 
TUCSON, ARIZONA 85734 


INTRODUCTION 

The Fabrication Information System was designed as an aid to Product Assurance 
personnel in the collection of quality data in various areas of the Fabrication Shop, 
in order that predominant problem areas be readily recognized. The purpose of the 
system is to: (Figure 1.) 

a. Collect yield data. 

b. Provide on-line visibility of defect data. 

c. Issue periodic reports to cognizant responsible personnel, summarizing defect 
data. 

d. Calculate and display Process Control Limits (PCL) for any part number in the 
system. 

e. Allow the on-line application of Process Control Limits based on the previous 
52 week period. 

f. Obtain appropriate corrective action as necessary, if the process exceeds the 
calculated limit based on the capability of the process. 


DATA AVAILABILITY AND RETENTION 



1. ON-LINE FIS File: Contains detail records of defective and acceptable parts. 

Data remains in file for 13 weeks after the work order processes 
to Stores. 


2. Weekly Tape: Same data as on-line file. 

3. Archive Tape: Contains records for parts whose work 

orders have closed to Stores more than 13 weeks ago. 

FIGURE 1 

The quality history information sources for the system files are the Fabrication 
Information Log (Figure 2) completed by the floor inspectors daily for all work orders 
processed through each inspection station, including the Material Review area. 
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FABRICATION INFORMATION SUMMARY 







Information concerning the quantity and configuration of the items in the work 
order moved between operations is supplied by the Fab Location System upon update by 
Production Control personnel. Defect information is recorded on the appropriate 
summary logs by Fab inspectors under the Quality Department direction and batch pro- 
cessed into the system files daily via various satellite CRT terminals strategically 
located within the facility. 

Data fed into the system is maintained in the on-line files, allowing interested 
persons to easily obtain history on a given part number and operation number. The on- 
line CRT screens may be updated too, for instance, add corrective action narratives 
to the files for a specific lot of parts. 

Control limits are directed under the cognizance of the Product Assurance Fab 
Engineering group and may be of three types: 

a. Percent fraction defective. (P) 

b. Defects per unit. 

c. Engineering action limits. 

Percent fraction defective control limits are developed utilizing available his- 
torical data to establish a long term average where X is the number of standard devia- 
tions in the formula: 

IJCL = P + A / P (1 - P) v 100 

V N 

Initial control limits were established based on 13 weeks of fabrication history 
after manufacturing techniques, methods, and tooling had been established. The 
machines used were dedicated to a specific aerospace program and were operated on a 
3 shift basis performing repetitive operations. The majority of the machines utilized 
were NC tape operated or controlled mechanically be fixed cams and templates to control 
the necessary machine functions for the machining operations. Various functions per- 
formed by the machines were turning, milling, drilling, tapping, and boring. Many of 
the drilling and tapping operations were "ganged" and all holes drilled and tapped 
simultaneously. Tolerances varied in the range of from .00005 for critical gimbal 
bearing bores up to +_ .010 for the numerous various characteristics. Many of the parts 
were machined from aluminum preforms, bar stock, or extrusions. Hand, and vibratory 
tumble debarring was performed. 

An excerpt from a typical EDP Summary Report (Figure 3) summarizing all of the 
part numbers fabricated for the past 52 weeks lj_sted by operation, lot size, quantity 
suspended, and the calculated process average (P) is updated each 13 week period. The 
responsible Product Assurance engineer reviews this data to determine which data should 
be discarded from the process average because of it not being normal to the manufac- 
turing process. Data eliminated includes machine/fixture malfunctions, operator error 
which affects a large portion of the work order before being recognized, tool breakdown 
or any variation not normal to the established manufacturing process. 
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75 
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FIGURE 3 



A report is then issued each 13 week period (Figure 4), listing corrected P which 
reflects the normal variation expected from the machining process. This tabulation is 
utilized by the Product Assurance floor engineer, in conjunction with the Product 
Assurance Allowance Table (Table I), to determine the necessity for corrective action 
as the discrepant material is reviewed. 


I’Atfl' NUHL>m 

Ol’EhAT LON. 

£ 

CONTUDL LIMIT % 




NEW OLD USE (WW 27-39) 

Hou sing , Elec t ronic 

20 

0.5 



30 

0.5 



UO 

0.5 



50 

6.0 

10.7 10.3 10.3 


60 

0.5 



70 

1.4 



80 

0.5 



100 

0.5 



110 

0.5 



120 

0.3 



123 

1.4 



130 

0.5 


Nose, Cover 




322722 5-1-1 

30 

0.5 



40 

0.5 



30 

2.1 



60 

0.5 



65 

0.5 



70 

0.5 



80 

0.5 



90 

0.5 


Girabal, Outer 




3227315-1-1 

20 

0.7 



30 

0.9 



30A 

1.7 



30B 

1.0 



40 

1.3 



50 

0.6 



60 

0.5 



70 

2.5 

5.2 5.2 5.2 


80 

0.9 



90 

0.5 



100 

2.8 



110 

6.7 

11.7 8.5 8.5 


120 

1.9 



125 

0.5 




FIGURE 
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TABLE i 

PRODUCT ASSURANCE ALLOWANCE TABLE 


THIS TABLE MAY BE USED FOR APPROVED CONTROL LIMITS ONLY 


LEVEL 




2 

<J- STANDARD DEVIATION 




P(S) 



ALLOWABLE 

PARTS DEFECTIVE/WORK ORDER SIZE 




2 

3 

4 

5 

6 

7 

8 

9 

10 

0.5 

34 

107 

200 

306 

421 

542 

669 

801 

936 

1074 

06 

28 

89 

167 

255 

351 

452 

558 

668 

780 

896 

0.7 

24 

76 

143 

218 

301 

387 

478 

572 

669 

768 

0.8 

21 

67 

125 

191 

263 

339 

419 

501 

585 

672 

0,» 

19 

59 

111 

170 

234 

301 

372 

445 

520 

597 

VO 

17 

53 

100 

153 

210 

271 

335 

401 

469 

538 

1.1 

15 

« 

91 

139 

191 

247 

305 

364 

426 

489 

1.2 

14 


83 

128 

175 

226 

279 

334 

391 

448 

1.3 

13 

41 

77 

118 

162 

209 

258 

309 

341 

414 

1.4 

12 

38 

71 

109 

150 

194 

239 

287 

335 

384 

1 5 

11 

36 

67 

>02 

140 

181 

224 

268 

313 

359 

1.6 

10 

33 

43 

96 

132 

170 

210 

251 

293 | 

337 

1.7 

10 

31 

W 

90 

124 

160 

197 

236 

276 

317 

1 8 

9 

» 

56 

85 

1 17 

151 

186 

223 

261 

299 

1.9 

9 

28 

53 

81 

lit 

143 

177 

211 

247 

284 

2.0 

8 i 

27 

50 ! 

77 

105 

136 

168 

201 

235 

269 

2.1 

8 

25 

48 

73 

100 

130 

160 

191 

224 

257 

2 2 

7 

24 

45 

70 

96 

124 

153 

183 

214 

245 

2.3 

7 

23 

43 

67 

92 

118 

146 

175 

204 

234 

2.4 

7 

22 

42 

64 

88 

113 

140 

168 

196 

225 

2.5 

6 

21 

40 

61 

84 

109 

134 

161 

188 

216 

2 6 

6 

20 

38 

59 

81 

105 

129 

155 i 

181 

208 

2.7 

6 

20 

37 

57 

78 

101 

125 

149 

174 

200 

2.6 

6 

19 

36 

55 

75 

97 

120 

144 

168 

193 

2 9 

6 

18 

34 

53 

73 

94 

l 16 

139 

162 

186 


USE 1 Loco*c the opp'Oved iong te f m overoge pe'centoge (T^! in me left hond column 


2 Find the A-crh s.se or the neo'est lower number u: der the ad-ocent right columns 

3 Ceterm.ne the of*c*obie pons defective ot the top of the column. 

P n=3 Long term overoge percent defective. 


L EGEND 




Since the reporting function for each operation includes all discrepant material 
including set-up scrap, it was arbitrarily decided that . 5 % defect per work order 
would be the minimum acceptable process average below which corrective action would 
not have to be taken. 

If there is no reason to believe that any of the manufacturing process has signif- 
icantly changed, F is a reasonable estimator of the fraction defective for the lot 
fabri cated . 

Since the sampling technique of the work orders sorts the nonconforming units 
taken randomly from the lot, it has at least one of the principal requirements of a 
binomial distribution. Samples contain acceptable or nonconforming pieces. Samples 
taken from successive lots using the same or similar manufacturing techniques will have 
a characteristic central tendency but will also have a characteristic dispersion. The 
more defectives, the wider the dispersion. As one might expect, the smaller the number 
of defectives in the lot, the narrower the dispersion. 

To assist the Product Assurance engineer on the floor in determining the need for 
corrective action when reviewing the Quality Status Tag which documents any discrepan- 
cies in the work order, it was necessary to devise a rapid method to know if the manu-. 
facturing process for that operation was in control based on the past manufacturing 
history. 

The Allowance Table (Table I) was devised by utilizing the basic equation for 
calculating the control limits for a P chart by substituting in the equation the per- 
missible number of defectives from 1 to 10 in any lot for any long term average percent 
defective. By solving the equation for N and then rounding, the lot size was readily 
determined for any P. The allowable parts defective chart versus the work order size was 
tabulated and was readily available for the supporting engineers use. The calculations 
were performed on the_computer and listed the maximum number of defectives allowed in 
a work order for any P. Any time the number of defectives in a work order exceeded 
that allowed by the table, the cognizant Product Assurance engineer would investigate 
to determine the exact cause and obtain the appropriate corrective action. Action 
limits were established at 2 standard deviations where corrective action was necessary. 

Since the reporting function readily distinguished the problem areas, a conserted 
effort was expended on the worst ones first to determine the cause of the repetitive 
discrepancies in order that the appropriate corrective action be taken. 

After obtaining an acceptable level, the operation with the next highest discrep- 
ancy rate was investigated until that operation was brought into control. Thus within 
a reasonable length of time, all operations were running with an acceptable percent 
defective. 

This Fabrication Information System has been in operation for approximately 3 
years. During this period of time, the data collected from this system has identified 
numerous potential/actual problem areas. These problems were investigated by the 
Fabrication Product Assurance Engineering organization and the cause of the problems 
were isolated. The responsible cognizant organization was then notified and the appro- 
priate corrective action was initiated. The data was monitored to assure that the 
corrective action was effective. 

A Fabrication Quality Report consisting of three sections is issued weekly. 

Section I contains the list of Part Number/Operation Number which will be reported on 
for the present quarter. Control limits are shown with their respective Part 
Number/Operation (Table II). Part numbers are reviewed quarterly and those which are 
designated by the responsible Fabrication Product Assurance engineer are included on 
the list. The Control Status column indicates the number of consecutive weeks the 
Part Number/Operation has exceeded its control limit 


TABLE II 


Part Number 

Oper . 
No. 

Qty 

In W.O. 

Qty 

Susp. 

7. 

Susp. 

Control 

Limits 

Control 

Status 

3227221-2-1 

050 

0 

0 

... 

1.9 


3422667-1-1 

057 

243 

0 

0.0 

2.3 


3422667-1-1 

060 

100 

0 

0.0 

1.2 


3422673-1-1 

020 

48 

0 

0.0 

1.1 


3422673-1-1 

030 

0 

0 

--- 

2.0 


3422673-1-1 

120 

98 

3 

3.1 

2.9 

OUT 

3422673-1-1 

130 

51 

0 

0.0 

1.3 


3422673-1-1 

135 

47 

0 

0.0 

3.2 


3422673-1-1 

160 

46 

0 

0.0 

2.4 


3422673-1-1 

180 

0 

0 

— 

1.6 


3422674-1-1 

150 

321 

3 

0.9 

1.1 



Section II, as a minimum, contains statement of cause and corrective action for 
all Part Number/Operations (Figure 5) which have exceeded their control limits for 3 
consecutive weeks and will be reported in the following weeks report, along with their 
trend charts. Normal corrective action is accomplished by applying the upper control 
limit on a case by case basis during the daily production activities. 


REF: TQP 1.5.101 Control Limits - Development and Applications 


The Part Numbers/Operations listed on the following pages Indicate an accept- 
able quality attrition level for operations being performed on Maverick 
missile fab Items In consonance with the requirements defined In the referent 
TOP. 

The attrition levels shown, generated by the Fabrication Information System, 
are long term fraction defectives (F) calculated from data between MW 14, 

1976 and WW 13, 1977 on a weekly basis with the corresponding calculated 3 (T 
control limit where applicable. The next reevaluation will be made WW 27, 1977. 

All operations show a decrease In the F or have remained at a low constant. The 
tabulation for P/N 3227580-1-1, Operation 105, indicates that since W/W 34, dis- 
crepancies have been at a minimum, reflected by the continued reduction In the 
control limit. Thus, the corrective action taken was effective. 

P/N 3240377 - This part has been completely re-planned. Therefore, operation 
67 and 68 have been removed from the report and replaced with operation 30 and 
60. 


Operation 50 (vibratory deburr) has been added to the report due to the high F 
figure (2.2S). 

CAUSE: Excessive burrs In slots. 

CORRECTIVE ACTION: Media has been changed to vlbro-cut pyramids and carpet 
tacks. 

Operation 50 will remain on the report to monitor the effectiveness of the correc- 
tive action. 

The^ tabulation, attached, lists the old 3 calculated control limits that were 
above' Si in addition to the new 3 f control limit for comparison. 


FIGURE 5 


Section III contains the trend charts for Part Number/Operation Number listed in 
Section I. Only trend charts where the Part Number/Operation are out of control will 
be included. (Figure 6.) 




FABRICATION TREND CHARTS 















The implementation of this system has contributed to a timely and effective 
method of identification of defects and the initiation of corrective action for a major 
missile program. This has resulted in a lower cost of the product because of the re- 
duction in scrap and rework costs. 

An effective corrective action program has many benefits other than the obvious 
one of reducing excessive scrap and rework costs. It improves schedule time, elimi- 
nates overhead charges such as handling and storage of nonconforming parts, reduces 
inspection labor (it is easier to inspect good parts than bad parts), reduces customer 
complaints, and motivates people to do their best work. 

As soon as first line supervision realizes that corrective action is helping them 
solve line stoppage problems and meet their schedules and budgets, a better relation- 
ship has been established between Manufacturing, Quality Control, Engineering, Material 
Control, Purchasing, and the other departments that convert raw materials into finished 
goods. The best indication that the program is working is when Manufacturing super- 
visors begin to solicit assistance from corrective action personnel. 
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EFFECT OF BIAS IN ESTIMATING PROCESS VARIABILITY ON ARL FOR CUSUM CHARTS 


C. P. Kartha, School of Business Administration 
University of Notre Dame, Notre Dame, Indiana 

and 

Bovas Abraham, Department of Statistics 
University of Waterloo, Waterloo, Ontario 


1. INTRODUCTION 

Most of the control chart procedures assume that the process variability is known 
or can be estimated with sufficient degree of accuracy from data already at hand. In 
practice the variance of the process is estimated using sample ranges and sample 
variances. One of the most frequently used method is to use the sample variance 

s2 ■ Mi] j/V ® 2 (ia) 

where n is the sample size and X, the sample mean, for estimating the process 
variance a 2 . As is well-known, S 2 is an unbiased estimator of o 2 under the as- 
sumption that the observations are independent and identically distributed random 

variables. In practice, however, the observations from many industrial processes are 
not independent but serially correlated. This introduces a bias in estimation of a 2 
and also affects control procedures in which the correlation among the observations 
will have an effect on their optimal properties. Goldsmith and Whitfield [5] studied 
the effect of serial correlation on the parameters of V-mask for cusum charts. A simi- 
lar study was done by Hunter and Kartha [6] on the optimum target values of a produc- 
tion process in which observations from Autoregressive and Moving Average models were 
considered. Also bias in estimating a 2 for these models was considered which will be 
discussed in Section 2 of this paper. In Section 3 we will study the effect of this 
bias on the parameters of a cusum chart under a decision interval scheme. 

2. BIAS IN ESTIMATING o 2 

We shall consider the bias in estimating the process variance a 2 by using (1.1) 
when the observations are serially correlated. In particular we will consider the 
situations in which the observations follow either a first-order Autoregressive pro- 
cess (AR(1)) or a first-order Moving Average process (MA(1)). (Box and Jenkins [2]). 

(i) AR(1) Process 


Suppose observations Xj, X 2 , 


are from an AR(1) process defined by 


(x t ~u) = d>(x t _ 1 -m) 


( 2 . 1 ) 


where 0 < | $ | <1 and (a t ) is a white noise series, normally distributed with mean 
0 and variance o|. The vector (X t ,X t _i, . . . ) will follow a normal distribution 
with mean (p,ii, . . . ) and variance covariance matrix Z = {ai-j} where 


and 


ij 


CovfX^Xj) = <J>- 


cr^ = var(X^) = a 2 


a-^rV 

a 


( 2 . 2 ) 

(2.3) 


n 


E(S 2 ) = E { — — Z (X -X) 2 } = ~{Z E(X 2 ) - n E(X) 2 } = (n(y 2 +a 2 ) - n E(X) 2 } (2.4) 

n-.L i n-1 1 n-i 

It can be shown that 


E (X 2 ) = V 2 + -°2 + 4 tfS: - > 

n n^ 1— <±> (1-dr-) 


(2.5) 
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( 2 . 6 ) 


ECS 2 ) 


J> , <i>(i- (j) a ) i 

( n- 1 j ( 1- <j> ) n(n-lj (l-$) 2 


It follows from (2.6) that S 2 is a consistent estimator of a 2 for | tj> | being not 
close to 1. If | tp | is near one, for moderately large n, (4.25) would tent to 
underestimate a 2 for 4>>0 and overestimate for <)> <0 . When <J>=0, S 2 is an unbiased 
estimator of a 2 . 


( ii) MA( 1) Process 

Suppose that the observations are froinaMA(l) process given by 


<X fc -y) = a t - 6a t _ 1 (2.7) 

where 0< | 0 | < 1 and {a t } is a white noise series as defined above. In this case, the 
vector (Xt,X t _i, . . . ) is normally distributed with mean (u,u, . . . ) and with 
variance- covariance matrix 7. = {a-j^} where 


and 


As before 




| -{0/(l+0 2 ) }o 2 if [ j-i j = 1 
0 if | j-i | > 1 


a . ,=o 2 = ( l+O 2 ) a 2 


E(S 2 ) = - (na 2 +u 2 ) - n E(X) 

n-1 


( 2 . 8 ) 


(2.9) 


and it is shown that 

E(X) 2 = ^2 (n(o 2 +u 2 ) + 2(n-l)p 1 a 2 + n(n-l)u 2 } (2.10) 

Substituting (2.10) into (2.9) and simplifying 

E(S 2 ) = o 2 (l-2 Pl /n) (2.11) 

where 

Pi = -6/(l+0 2 ) (2.12) 

It follows from (2.11) that S 2 will be overestimated for 0>O and underestimated for 
0<O. For large n , E(S 2 ) = a 2 . 

3. EFFECT OF BIAS ON ARL OF CUSUM CHARTS 

The use of cumulative sum control charts, as an alternative to standard control 
charts was introduced by Page [8], Barnard [1] discussed their construction and use 
in quality control and introduced an elegant graphical procedure. By this method a 
V-shaped mask is superimposed on the cusum chart, the vertex pointing horizontally 
forwards and set at a distance d ahead of the most recent point. The angle between 
the obliques and the horizontal is denoted by 0. If all the previously plotted points 
fall with V the process is assumed to be in control. Page [8] and Ewan and Kemp [4] 
discussed an equivalent numerical procedure by which cumulative sums are obtained on 
standardized deviations minus a reference value k. The effect of subtracting a ref- 
erence value is to produce a downward slope of the cusums when the process is in con- 
trol. An out-of-control signal is triggered as soon as this biased cusum graph exceeds 
its minimum by more than a specified amount h , called the decision interval. The 
relationship between the parameters d and 0 of the test using the V-mask and the 
parameters k and h of the decision interval scheme is given by h = wd tan 8 and 
k = w tan 0 where w is the scale factor. Extensive tables for Average Run Lengths 
for various values of h and k are available for independent normal variates. 

(Ewan and Kemp [4], DeBruyn [3]). Goldsmith and Whitfield [5] gave an extensive set of 
graphs of the ARL as a function of the V-mask parameters d and 0 ising Monte Carlo 
methods. They also studied the effect of serial correlation among the observations on 
the ARL and provided two graphs showing ARL as a function of the serial correlation 
coefficient and 6 , the displacement in the mean desired to be detected. Using their 
graphs an equivalent decision interval scheme with h = 2.0 and k - 1.0 can be 



.9 


5.263 


3.120 

(-40.7) 


3.42 

(-34.' 


0 




-.9 

1.810 

1.450 

(- 20 . 0 ) 

1 . 63 < 
(- 10 . ( 
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1.250 

1.050 
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0 
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.9 

1 . 810 
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itected. Using their graphs an equivalent dccisi< 
1.0 can be derived for an AR(1) model (2.1) witl 
‘Cted values of <f> as given in Table II below, 
ndence on the ARL on <J> except when <f» is closi 
ie effect is to decrease the ARL when 4> is posi 
‘gative . 


Table II. ARL 


a 


function 


<f> 


6-k 

.9 

.5 

0 

-1.0 

50.0 

46.0 

130.0 

-0.5 

36.0 

23.0 

49.0 

0 

23.0 

6.0 

9.0 

1 

6.1 

3.1 

3.0 

2 

2.0 

1.6 

1.6.0 


Suppose the variance of the AR(1) process is 
:andardized values are used in a decision interva 
set of over and under estimation of the standard 
irameters effectively by that same factor. Using 
>r selected values of n modified values for h 












calculated individually for each combination of n and 0 . For positive values of 0 
the effect of the bias was to reduce ARL while for negative values of 0 the effect 
was to increase ARL with extreme variations when | 0 | was close to 1 and when n is 
small. The effect of bias was more significant for positive values of 0 compared to 
negative values of 0 . In fact the effect of serial correlation and bias in estima- 
tion of the variance on ARL is minimal for moderate values of 0 and n and in such 
cases existing tables for independent normal variates can be used for selecting cusum 
test parameters. 


REFERENCES 

1. Barnard, G. A. (1959), "Control Charts and Stochastic Processes", U.R.S.S., B, 
21, 239. 

2. Box, G.E.P. and Jenkins, G. M. (1970), Time Series Analysis - Forecasting and 
Control, Holden Day, San Francisco. 

3. DeBruyn, C. S., (1968), Cumulative Sum Techniques, Monograph. 

4. Ewan, W. D. and Kemp, K. W. (1960), "Sampling Inspection of Continuous Processes 
with no Autocorrelation between Successive Results", Biometrika, 47, 363. 

5. Goldsmith, P. L. and Whitfield, H. (1961), "Average Run Lengths in Cumulative 
Sum Chart Quality Control Schemes", Technometrics, 3, 11. 

6. Hunter, W. G. and Kartha, C. P. (1975), "Effect of Serial Correlation on the 
Optimal Target Value of a Production Process", A.S.A. Proceedings, Atlanta, 
August 1975. 

7. Johnson, N. L. (1961), "A Simple Theoretical Approach to Cumulative Sum Charts", 
J.A.S.A., 56, 835. 

8. Page, E. S. (1954), "Continuous Inspection Schemes, Biometrika, 41, 100. 


LCS 114:20:400 



DISCOVERY SAMPLING IN AN AUDIT SITUATION 


Charles A. Mills, F.I.E.E., P.Eng. 
Manager, Quality Assurance 
Electronic Systems Division 
Westinghouse Canada Limited 


INTRODUCTION 

A Quality Audit is defined as a systematic examination of the 
acts and/or decisions of people with respect to quality, verified by 
the examination of both methods and results. Therefore, it can be 
used to determine the suitability of a quality programme and/or the 
conformance to a quality programme. 

Of necessity, the examination of methods and results must involve 
some form of sampling techniques. This application of sampling tech- 
niques differs somewhat from the normal sampling procedures, in that 
the auditor is not basically concerned with making an acceptance or 
rejection decision. In the audit situation, the auditor must be able 
to state that the operations do or do not conform to the quality 
programme or that the quality programme is or is not suitable. 

PURPOSE 

The purpose of a Quality Audit is to discover if any operational 
deficiencies are present and, if they are present, the extent of these 
deficiencies. This type of sampling called 'Discovery Sampling’ in 
accounting circles has not received adequate attention in the quality 
audit field. 


DISCOVERY SAMPLING 

In Discovery Sampling, the basic factors that determine the 
sample size are: 

1. The number of departures from the quality programme which 
are permitted, 

and 

2. The risk of failing to discover the departures. 

The first factor to be determined is the number of departures 
from the programme which can be tolerated. This factor can vary from 
0 to quite large numbers depending upon the nature of the Quality 
Audit. If the audit is covering a particularly critical function such 
as Inventory Control in a nuclear fuel facility, major safety or legal 
controls, then the number of departures which can be tolerated would 
be 0. In other areas having lesser system significance various numbers 
or percentages of departures could be tolerated. 

The second factor to be determined is the risk of failing to 
include in the sample, examples of any departures present. This 
factor can be expressed either as the risk of failing to discover 
deficiencies, dr alternatively, the confidence level that examples of 
the deficiencies have been discovered, if present. 

In most Quality Audit situations confidence levels of 90% - 95% 
would normally be used. Tn extremely critical situations, such as 
those permitting no departures from the control programme, the confi- 
dence level would, of necessity, be much higher, e.g. 99% or greater. 
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SAMPLING PLANS - STANDARDS 


Currently, no standards exist covering the sampling techniques in 
a Quality Audit situation. Some guidelines have been developed in the 
accounting circles. However, these do not have the recognition which 
would be given to a formally issued standard. 

With the rapidly developing importance of the Quality Audit in 
many industries, I believe that the Standards Committee of the ASQC 
should undertake to develop and promulgate a Standard covering the 
various acceptable sampling plans used in an audit situation. 

SAMPLING PLANS - REQUIREMENTS 


A Society Standard on 'Discovery Sampling 1 would provide a found- 
ation for a professional approach to Quality Audit. By the Society 
taking a lead in this important facet of the Quality Audit, the Society 
can demonstrate its concern with the Quality Audit. 


In developing a standard there are two basic audit situations 


■ sampling plans are 

required : 





1 . Those in w'hich 

of precision. 

the 

popul at ion 

is known, 

with 

a fair 

degree 

2 . Those in which 

of precision. 

the 

population 

is not known, 

with any 

.degree 

In both situations, 

it 

will be necessary to 

have 

sampling 

plans 


which can relate 


1 . The number of departures from the quality programme which 
are permitted. 

2. The risk of failing to discover examples of these departures. 

In developing the Sampling Plans, cognizance should be taken of 
the need to know' the discrimination present in the Sampling Plan. 
Discrimination can be defined as the range of error rates lying 
between the upper and low'er risk points for the sample size concerned. 
It, therefore, provides a measure of the band of error rates identified 
by the findings of the Quality Audit. 

SAMPLING PLANS - IN PRACTICE 

In practice, three approaches to Sampling Plans in a Quality 
Audit situation are commonly used: 

1. Ilypergeometric Sampling Plans applying in situations where 
the population is known, with a fair degree of precision. 

2. Binomial Sampling Plans applying in situations where a 
limited number of departures from the control programme are 
permitted and the population is not known, with any degree 
of precision. 

3. Lot Tolerance Percent Defective applying in those situations 
where a somewhat larger percentage of departures from the 
control conditions are permitted and the population is not 
known, with any degree of precision. 

Various general Sampling Plans are in existence covering each of 
these three types of plans. However, their interpretation and appli- 
cation to audit situations will vary from auditor to auditor. 



CONCLUSION 


I believe that a Standard on recognized Sampling Plans for audit 
situations is required within the National Standards of both United 
States and Canada. With the ASQC involvement with the ANSI Z-l Committee 
and its liaison activity with the Steering Committee on Assurance 
Sciences of the Canadian Standards Association, the Society is in a 
unique position to lead in the development of a Standard covering 
Sampling Plans for Quality Audits. 

Some activity is currently underway on this topic in the Technical 
Committee on ’Application of Statistical Techniques' of the Canadian 
Standards Association. Currently, this work is directly involved with 
the audit of the Inventory Control on nuclear fuel. However, its 
activities will be broadened to cover the total audit situation. 
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ANOTHER LOOK AT L.Q. SAMPLING 


David Pettigrew, C.Q.E. 
Grayhill Inc. 

La Grange, IL 60525 


The purpose of this paper is to help those who must use sampling procedures and 
tables -for Inspection by Attributes, contained in Mil-Std-105. This will also in- 
struct them on some of the uses of Limiting Quality (LQ) , for isolated lot, sample 
inspection, and to reduce overall inspection costs. 

The purpose of inspection is to assure that the process is, or has, performed 
to the best of its capability. We look for rejects only when a process is known to 
be a producing defective product at a specific, lower than its capability, defect 
rate. Under normal inspection conditions, Sampling Procedures are used to reduce 
inspection errors: Bias; Fatigue; Monotony; Etcetera. Sampling tables are based on 
the laws of chance, (probability laws) and when used in accordance with these laws 
they will afford excellent protection against accepting a lot with a given percent 
defective, or rejecting a lot of Acceptable Quality Level (A.Q.L.). 

Most organizations have adopted Mil-Std-105, "Sampling Procedures and Tables for 
Inspection by Attributes." Although this is the most widely accepted sampling pro- 
cedure, or perhaps because of its wide acceptance, it is probably the most misused 
Military Standard. 

As stated in the second paragraph, "The purpose of inspection is to assure that 
the process is, or has, performed to the best of ITS capability." In almost any pro- 
cess, from the simplest to the most complex, there is always a risk of producing a 
defective product. Quality Control and Inspection are charged with the responsibility 
of controlling that risk. The most economical method to control the risk is by Sample 
Inspection, performed correctly and accurately. 

Why do we use sample inspection rather than 100% inspection of all products? The 
most obvious answer is: "Sample inspection takes less time and is therefore more ec- 
onomical." Second, suppose we were inspecting rifle shells with 100% inspection, we 
would end up with no product to ship. Finally it has been proven that even with 100% 
inspection, the Inspector will accept a portion of any defective material submitted 
for inspection, mainly due to Inspection error; Observation error; Bias; Manipulative 
error; Fatigue; Monotony; Boredom; Inadvertent error; and many others. 

There are two general methods of inspection in use today, namely: Attribute and 
Variable inspection. Attribute inspection is primarily used to determine whether a 
characteristic exists, with respect to a given requirement, (Go and No-Go inspection), 
as an example in a push button switch, we inspect to assure that the contacts open or 
close to make or break an electrical circuit. 

Attribute inspection is most often used in examination for visual defects, miss- 
ing or existing operations and/or characteristics, workmanship defects, incorrect 
dimensions or other numerical values expressed as an accept or reject criteria (such 
as gaging with a pre-set, fixed gage). Attribute inspection is used to determine 
solely whether an item does or does not meet the established requirements without 
regards to actual numerical values. 

The advantages of attribute inspection are many, among which some of the more 
important are: Cost, and Simplicity and Speed of inspection. The primary disadvan- 
tage is: Very little (if any) numerical value is known about the item under test, 
and therefore adjustments to the process are made on an intuitive basis. The amount 
of change required may not be fully known. In many cases this type of information is 
all that is required. That is, as an example: In punching operations, if an excess- 
ive burr exists, we may not need to know the exact dimension of the burr to know that 
the punch and/or die need re-work or sharpening. 
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Variable inspection is used to determine the precise point at which a character- 
istic measures. Example: Linear measurement of a length in thousandths of an inch, 
or the resistance of an electrical connection in Milli-Ohms. These may be converted 
to attribute inspection by using minimum and/or maximum values (limits) and disre- 
garding any and all intermediate values. This is often performed by Go, No-Go gaging, 
such as thread ring and plug gages. We do not determine any numerical values, only 
conformance to the gage characteristics. 

Items are usually submitted for inspection in one, or a combination of the fol- 
lowing methods: Continuous production flow; or separated into lots and submitted as 
lot by lot, or isolated lot inspection. 

Continuous production flow, is examination of items (usually 100%) as they are 
manufactured. This type of inspection is best suited for automatic inspection, such 
as sorting operations, either as good or bad, or into specific graduated measurement 
categories . 

The most common inspection is lot by lot, either as completed, continuous, or 
partial lots. This is where common sense analysis is of prime importance. Procedures 
and tables must be used properly, to evaluate the approximate quality level of the 
item being inspected. The sampling procedures and acceptance level must be based upon 
proven statistical practices. The sample and rejection numbers must reflect the de- 
sired protection to both the producer and the consumer. Many of the lot by lot in- 
spection plans should be calculated using "Isolated Lot Procedures." The term Iso- 
lated Lot actually refers to those sampling plans where the Limiting Quality (L.Q.) 
and consumers risk are applied. The lots do not have to be isolated in. the physical 
sense before applying these plans to the concepts of sampling inspection procedures. 
Limiting Quality is "The Worst Quality Level that you are willing to accept a portion 
of the time." This may seem to be a rather harsh statement, such as: "I am willing 
to accept 4% defective, 5% of the time." What I have actually said is: "If the pro- 
cess is producing at an average of 4% defective, I am willing to accept only one (1) 
lot of twenty (20) lots submitted. The remaining nineteen (19) lots will be rejected, 
depending on the process average remaining at 4% defective. If the process average 
changes all various percent defective lots will have a pre-determined chance of ac- 
ceptance. This chance may easily be determined by using the tables and data found in 
Mil-Std-105 . 

With Limiting Quality Inspection, lot by lot sampling, the average outgoing Qual- 
ity Limit (A.O.Q.L.) may be determined from the A.Q.L. of the sampling plan. In our 
example of L.Q. of 4% defective 5% of the time. We arrive at an A.Q.L. of 0.65 (using 
a sample size of 200 pieces). The A.O.Q.L. for this plan, from table V of Mil-Std-105 
is 0.97. Note: For the exact A.O.Q.L. this factor must be multiplied by; 

Sample Size 

(1 - Lot Size ). If we assume the lot size to be 5,000 pieces, the exact A.O.Q.L. 

200 

is 0.9312 (1 - 5,000 X 0.97 = 0.9312. If our lot size is 10,000 pieces the A.O.Q.L. 

200 

is 0.97 X (1 - 10,000) = 0.9506. This is assuming all defectives from the rejected 
lots are removed and replaced with acceptable product. 

Let's take a closer look at L.Q. type of Acceptance Sampling. First off, Limit- 
ing Quality (L.Q.) is defined (per Mil-Hdbk-53) as "The Worst Product Quality that the 
consumer is willing to accept." I believe this has lead most of us to believe large 
sample sizes and therefore high inspection costs. This, as I will try to prove, may 
not always be true. 

Sampling plans may be devised to provide a specific Limiting Quality protection 
to the consumer. These plans should be used with "Isolated Lot" (one time or inter- 
mittent production) where very little or No control is made over the process. Plans 
of this type are designed primarily to provide protection to the consumer. 

I will explain a simple plan that can give equal or better consumer protection 
for a 2.5% A.Q.L. if our average sample size has been 200 pieces. With this plan we 
accept with 10 rejects and reject on 11 defects. About 5% of the time we may accept 
a lot as bad as 8.48% defective. 



Now if I consider this to be acceptable, I can state that "I am willing to accept 
8.5% defective 5% of the time. I can now go to table VII-A of Mil-Std-105 and choose 
a plan or plans which will give me this protection. There are, in fact, four (4) 
plans which give this approximate protection: These I have listed below: 


SAMPLE SIZE 

REJECTION NUMBER 

(Pa) PROTECTION 

32 

1 

8.9% 

80 

3 

7.7% 

125 

6 

8.4% 

200 

11 

8.5% 


Any of these will give the stated protection as shown: At a 5% level. I am now 
suggesting a double or triple sampling plan using the above listed sample sizes and 
rejection numbers. 

The procedure would be as follows: An initial sample of 80 pieces shall be drawn 
from each lot and examined for conformance to specifications. If two (2) or less de- 
fectives are found, the lot shall be accepted. If three to five (3 to 5) defects are 
found (of the same characteristic) an additional forty-five (45) pieces shall be drawn 
for a full sample of 125 pieces. If a total of five (5) or less defectives are found 
in the full sample of 125 pieces, the lot shall be accepted. If six (6) or more de- 
fectives are found, the lot shall be rejected, (by the inspector). 

Sampling plan for Limiting Quality Protection of 8.5% defective at a 5% consumer 

risk: 


PLAN: Initial Sample 

Defectives : 

0-2 
3-5 
6 - Over 

Full Sample 

Defectives : 

0-5 
6 - Over 


80 Pieces 

Disposition: 

Accept the lot 
Draw full sample 
Reject the lot 

125 Pieces 

Disposition: 

Accept the lot 
Reject the lot 


This sample plan uses an initial sample of 80 pieces, and a full sample of 125 
pieces. This plan may be used regardless of lot or batch size, with the following 
conditions applied: 1) The sample must be drawn as randomly as possible; 2) Each 
lot or container of over 3,000 pieces shall be subjected to examination, each con- 
tainer must have its own sample drawn and inspected per the sampling plan criteria 
as follows: 


CONTAINERS IN LOT: CONTAINERS TO BE SAMPLES: 

2-15 2 

16-25 3 

26-90 5 

If any of the sample containers contains more than six (6) defectives, all con- 
tainers shall be subject to sample inspection, at the discretion of the responsible 
party and/or parties. 3) When a lot of items is submitted in more than one contain- 
er and the total of all containers is less than 3,000 pieces, the containers may be 
combined to fulfill the 3,000 piece requirement, but must not exceed 6,000 pieces, 
for the combined total of all containers in the lot. 

You may want to use one of the other plans such as: 32 pieces in the initial 
sample and 80 pieces for the full sample. This could be used for reduced inspection, 
for known good vendors. You may want to use 125 piece initial sample and 200 piece 
full sample for tightened for vendors of doubtful quality. In any case the choice is 
yours and you may reduce inspection costs without giving up your desired protection. 



lyfO AbUt, IfcUMNlUAL UUINf tMtNUt I nAFNbAU i lUNb — UrllUAoU 


679 


With Limiting Quality (L.Q.) plans the reduction in inspection costs can be sig- 
nificant and your protection may be increased. With L.Q. plans your good vendors will 
be rewarded and your poor vendors will be more easily identified, at less cost. 

I would definitely recommend a dual or multiple sample plan using Limiting Qual- 
ity. From my own experience I have found we have inspected about 33% more lots with 
the same number of inspectors, and have had no adverse effect on our Quality for the 
past six months. This has resulted in a cost reduction of about 25% in our receiving 
inspection department. 

In all inspection there are certain risks inherent with inspection. In sample 
inspection there is the additional risk of human performance. A special risk attrib- 
uted to bias and "Luck of the Draw." These may result in false decisions, relative 
to "Good" and "Bad" lots. In sampling inspection there is always a chance, or risk, 
that good lots may be rejected, or bad lots accepted. Generally, the smaller the 
sample size the greater the risk. Since risks are inherent, to all inspection, and 
more so to sample inspection these risks must be clearly understood by all who use 
sampling plans and procedures. These risks may be explained as follows: Assuming a 
lot of items, with a given percent defective is submitted for inspection, at the in- 
spection station: What are the odds of accepting or rejecting the lot? This may be 
determined by the tables, data, and Operating Characteristic curves given in Tables 
X of Mil-Std-105 . 

Due to the variations of the sample (human performance) the sample results may 
lead to incorrect acceptance or rejection of any given lot. That is; a sampling plan 
may result in a small percentage of good lots being rejected, this is referred to as 
the suppliers or "Alpha" risk; and likewise, the sampling plan may accept a small 
percentage of bad lots, this is referred to as the consumers risk or "Beta" risk: 

These are afore-mentioned and are predictable by the tables and graphs in Mil-Std-105. 
If you desire to calculate some specific risk, not given in Mil-Std-105, these may be 
calculated using the terms of Poisson's exponential binomial limits. You may use in- 
terpolation of the Mil-Std-105 tables, for a close approximation. 


Conclusion and Summary: Quality Control is charged with the responsibility of 
reducing overall cost and in the Electro-mechanical business our biggest cost is In- 
spection. If we can reduce this cost and still "Assure that the process is, or has, 
performed to the best of its capability", we shall have made a significant contrib- 
ution to our organization. 


The appendix is taken from Mil-Std-105, and is shown here to aid in the use of 
these tables. 

Table VII A shows 4 sampling plans which may be used for a 8.5% consumers risk 
at the 5% level. These plans correspond with A.Q.L.'s as shown. For example: A 
sample size of 32 pieces corresponds to a 0.40 acceptable quality level and a sample 
size of 200 pieces corresponds to a 2.5 A.Q.L. 

Table II A shows the rejection number associated with each plan. 

Table V A shows the average out-going quality limit associated with each plan. 
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SUPPLIER AWARD OF EXCELLENCE 
A SUCCESSFUL PROGRAM 


Joseph J. Zeccardi, Manager Quality Assurance Engineering 
Mark A. Ransco, Manager SQA Planning and Coordination 
Xerox Corporation 

Joseph C. Wilson Center for Technology 
Webster, New York 14580 

ABSTRACT 

The Xerox Supplier Award of Excellence Program was initiated in 1970 to recognize 
and honor suppliers with outstanding quality performance. The original supplier 
selection criteria was based strictly on quality performance as measured by Receiving 
Inspection records. The current program, changed in 1973, is based on such factors 
as delivery, technical competence, cost consciousness and service, in addition to 
quality performance. These additional measures of performance upgrade the original 
quality award to a more comprehensive award of performance excellence. This paper 
describes the characteristics of the program which have made it a successful manage- 
ment tool at Xerox. 

INTRODUCTION 

April can rightfully be called Supplier Recognition Month at Xerox Corporation. 
Since 1970, Xerox has been recognizing its top suppliers in April with a formal award. 
This year was no exception. Xerox presented its Award of Excellence to 66 of its top 
suppliers for the outstanding work they did for the company in 1976. For 26 of those 
companies, this was the second consecutive year they earned the coveted award. One 
company has received the award each year since its inception. 

Fewer than one percent of the corporation's suppliers qualify for the annual 
award which is based on such factors as quality, delivery, technical competence, cost 
consciousness and service. The suppliers are selected by an awards selection commit- 
tee which is made up of two members from Supplier Quality Assurance and five members 
from. Mater ials Management. A chairman is selected each year by Materials Management. 

MEASURE OF SUCCESS 

The success of any program of this type is difficult to measure. However, the 
supplier repeat rate is considered a good measure of success. From the 1976 awards, 
the following is the award scorecard: 


First 

time 

winners 

45% 

Second 

time 

winners 

38 

Third 

time 

winners 

10 

Fourth 

time 

winners - 

5 

Fifth 

time 

winners 

2 


Sixty-five percent of the suppliers who received the award in 1976 were repeat 
performers. This level of repeat performance is an indication of how successful the 
program is at Xerox 

SELECTION PROCESS 

The selection process begins by the end of December when Supplier Quality Assur- 
ance selects the top 200 suppliers who have, delivered at least thirty (30) lots during 
the year and have, achieved a 90% or better quality record. In addition, the supplier 
must be an S.Q.A. Survey Approved supplier. From this list. Materials Management 
(Procurement) eliminates those suppliers with poor delivery records. The resultant 
list is then submitted to each Procurement Sub-section manager for the final list. 
Procurement and Supplier Quality Assurance are then each allowed a very limited 
number of "wild card" choices. Such choices must be documented and justified and 
they are subject to review and approval by the selection committee. 
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NOTIFICATION 

When a supplier is finally chosen. Procurement notifies him by mail. This is 
usually done early in the month that the award is presented. The letter of notifica- 
tion informs the supplier that he has been selected for the award, and it invites the 
supplier to attend and identify who will be present at the Awards Banquet to receive 
the award. Each supplier usually brings two to three people. 

AWARDS BANQUET 

The Awards Banquet is normally held at a hotel in Rochester, New York. Xerox 
Procurement and Supplier Quality Assurance divide the cost of the dinners, refreshments, 
table arrangements, other hotel related costs, the cost of the awards presented to the 
supplier and the plaque for Xerox listing the names of all suppliers who received the 
award. The supplier is expected to pay for his travel, room and meals other than the 
Banquet. In 1976, these costs were less than ten thousand dollars ($10,000). 

OTHER FACTORS 

There are no restrictions on how suppliers may use the Xerox award "internally", 
that is, within their own company newspapers or other vehicles of communication. How- 
ever, there is a Xerox policy which requires award winners to obtain prior Xerox 
approval to use the award in any public news release. 

TESTIMONIALS 

This is not the place to identify and quote specific testimonials, but it is 
important to make some general comments from the testimonials received from award 
winners. Their general theme is that (1) they are pleased to have been recognized 
to receive the award, (2) they are proud to display the award, (3) they feel that the 
award reflects a high level of quality standards, and (4) the award motivates them to 
maintain a high level of performance standards. 

CONCLUSION 

We at Xerox believe that the Supplier Award of Excellence Program works. That 
is, it recognizes those suppliers that meet objectives, it motivates suppliers to 
maintain a high level of performance over time, and it brings quality, cost and 
delivery oriented people closer together to improve upon previous years performance. 
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COMPUTER DESIGN OF OPTIMUM VARIABLE SAMPLING PLANS 


Merle D. Schmid, Ph.D. 

Director, Bachelor of Engineering Program 
University of Dayton, Dayton, Ohio 45469 

ABSTRACT 

The University of Dayton, School of Engineering, has developed a series of inter- 
active computer programs that solve most of the traditional and many special quality 
control problems. This paper shows practical applications for eight of these programs 
(SL.F.DEF, SL.2P.0C, SL.0C.5, DL.2P.FD,SU.2P.0C, DU.2P.FD, SXB.OC, and DXB.OC), all 
associated with variable sampling plans. Since each program is written in FORTRAN IV 
and is as computer-independent as possible, each is easily compilable on most companies' 
computers. Further, since all programs are interactive, they may be used by any 
quality control person who has access to a teletype terminal. 

The paper discusses how to design and evaluate variable sampling plans for special 
conditions not covered in standard sampling schemes such as MIL-STD-414. Actual 
examples are given, showing step-by-step procedures to design and/or evaluate sampling 
plans that meet defined producer and consumer risks for both fraction defective and X 
plans for either single or double specification limits and with o.r without known sigmas. 

INTRODUCTION 

This paper discusses eight interactive computer programs, all related to accep- 
tance sampling by variables. These programs are divided into three groups: (1) plans 
similar to MIL-STD-414, Section D, to control fraction defective where sigma is known 
(SL.2P.0C, DL.2P.FD, and SL.0C.5); (2) plans similar to MIL-STD-414, Section B, to 
control fraction defective where sigma is unknown (SU.2P.0C and DU.2P.FD); and (3) 
plans to give assurance regarding the mean of a process or lot (SXB.OC and DXB.OC). 

One additional program, "Single Limit Fraction Defective" (SL.F.DEF), is included to 
define and illustrate the effect of changes in X and O on the fraction defective of a 
process controlled by a single limit (either lower or upper) SDecification. This 
latter program is primarily descriptive and useful in simulating conditions in industry 
to aid management made decisions concerning changes in processes. Further, the print- 
outs from it provide a convenient means to introduce the subject of acceptance sampling 
by variables; therefore, it will be discussed first. 

Normally Distributed Quality Characteristics 

In almost all cases, available variable sampling plans assume that the quality 
characteristic being inspected is normally distributed in the underlying population 
(i.e., the lot or process being sampled). This assumption is usually close enough to 
the actual condition so that variable sampling plans based on it are useful in analyz- 
ing quality in real-world situations. All of the computer programs discussed in this 
paper use this assumption of normality, although one program (DU.2P.FD) does use the 
"t" distribution to compensate for small sample size. 

The program "SL.F.DEF" assumes that the quality characteristic of interest is 
normally distributed and that approximations of X and a can be selected that are 
reasonable and useful in analyzing the real-world process in which we are interested. 
The capabilities and value of this program are best illustrated by example. 

PROGRAM "SINGLE LIMIT FRACTION DEFECTIVE" (SL.F.DEF) 

Consider first a problem of process control at a dairy. One of the products 
packaged by this dairy is milk with a guaranteed minimum of 2 percent butterfat. Gov- 
ernment regulation states that not more than 2-1/2 percent of the lots selected at ran- 
dom and tested may be less than 2 percent butterfat. Program SL.F.DEF requires dimen- 
sional quantities as input:; therefore, it is necessary to convert the lower limit value 
of 2 percent butterfat to a dimensional value such as ounces per quart. This is easily 
done as follows: 

(0.02) (32) = 0.64 oz/qt. 

Let us assume that prior experience has shown that when processing small batches the 
average butterfat content needs to average 0.80 oz/qt or more, if almost all batches 
are to test out at 0.64 oz/qt or above. This average excess of 0.16 oz/qt of butterfat 
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is expensive, and it has been suggested that the variation between lots could be 
reduced if the milk were handled in larger lots and more thorougly blended. 

Computer input and output of program SL.F.DEF for this problem are shown in 
Figure 1. Convenient ranges in oz/qt were selected for X (0.72 to 0.80) and for a 
(0.04 to 0.08). As a study of Figure 1 shows, the computer program is interactive and 
asks the operator a series of questions which leads to an understandable printout for 
the process in question. 


EXEC SL.F.DEF 
* P500 OOOjSLED. 

Fortran iv program slfd started — 12/23/77 

please post a title fdr the study <ao characters or less;. 
»BUTTERfAT in 2% MILK 

please post the dimensions of the study uo characters or less). 

♦OUNCES BUTTERFAT P£R QUART (OZ/QT) 

WHICH SINGLE SPECIFICATION LIMIT upper OR LOWER (U,L) ? 

n 

PLEASE INPUT the LOWER SPECIFICATION LIMIT 
*0.6AO 

please input 5 values of sigma separated by commas 
.05/ .06, .07, .08 

Please input s values of xbar separated by commas 
*.72/.7<,^.76^.78>.80 

D( YOU WISH to DIRECT THE OUTPUT TO THE SYSTEM PRINTER? (Y,N) 



FIGURE 1 — Input and output of program SL.F.DEF for a 
process with lower specification limit. 

Figure 2 was prepared as an aid in explaining the significance of two values 
(0.02275 and 0.10565) in Figure 1 [note items (1) and (2)]. The "fraction defective" 
section of Figure 1 as well as the location of critical points on Figures 2, 3, and 4 
were developed using the "area under the tail" from a standard normal distribution area 
versus "z" table. The calculations are included as a subroutine within program SL.F. 
DEF. For example, the standard z value for item (2) in Figure 1 was calculated as 
shown in Equation (1) . 

z = (X ± - X)/a = (0.64 - 0.74)/0.08 = -1.25. (1) 

For a z value of -1.25, a standard normal distribution table shows the area under the 
left tail to be 0.10565. 





FIGURE 2 — Increase in percent defective FIGURE 3 — Decrease in percent defective 
by decreasing the mean (X) of by reducing the variation 

a process with lower specif i- (a) of a process, 

cation limit. 

Item (1) in Figure 1 (i.e., 0.02275) and the corresponding value (2.3 percent) at 
the left tail of the distribution of process (1) in Figure 2 illustrates the operating 
conditions assumed above for the milk processing plant (X = 0.80, C = 0.08). 



iilillf 


ioiu i Lvniiiwnu v/vm i-n i_in v_#t_ i r.Hl'tOHU I IUINO — ^niWHUV 


PSD 


If the dairy were to reduce the average quantity of butterfat (X) from 0.80 to 0.74 
oz/qt without doing anything to reduce the variation in the process (a = 0.08), then 
10.6 percent of the lots would test out as being below the lower limit of 0.64 oz/qt of 
butterfat [note item (2) in Figures 1 and 2] . 

It should be possible to reduce variation in the butterfat content of the milk by 
processing it in larger lots and blending it more thoroughly. For example^ if a were 
reduced from 0.08 to 0.05 oz/qt, then the average quantity of butterfat (X) could be 
reduced from 0.08 to 0.74 oz/qt, without increasing the number of rejected lots above 
the 2-1/2 percent regulation. This condition is illustrated as item (3) in Figure 1 
and distribution (3) in Figure 3. 


In most real-world situations, it is not possible to economically reduce the 
variation of a process to accomplish the desired results as illustrated above between 
items (1) and (3) in Figure 1 and between distribution (1) in Figure 2 and distribution 
(3) in Figure 3. 



FIGURE 4 


In most cases a compromise is necessary. 
In this case such a compromise might be to 
reduce the variation of the process some- 
what and at the same time use_a tradeoff 
of allowing a higher average X. This com- 
promise is illustrated as items (4) and (5) 
in Figure 1 and distributions (4) and (5) 
in Figure 4. Here the change between con- 
dition (4)_and (5) illustrates both a 
change of X from 0.72 to 0.76 and a change 
of 0 from 0.08 to 0.06. This simultaneous 
increase of X and decrease of o results in 
a reduction of the lots below the lower 
limit from 15.9 to 2.3 percent. This is 
typical of real-world compromises that are 


Decrease in percent defective 
by both increasing the mean and 

decreasing the variation of a , . 

. r . necessary to bring a process into control, 

process with lower specification y & * 

limit. As can be seen from Figure 1 and the 

explanation of this figure in Figures 2,3, 
the program SL.F.DEF can be used to simulate conditions in a real-world pro- 
It is possible using the printout of program SL.F.DEF to forecast the effect of 
changes in X and 0 on the_fraction defective of a process in question (i.e., the per- 
cent of the lots with an X below the lower limit) . 


and 4, 
cess . 


Program SL.F.DEF can also be used for processes that have a single upper limit. 
This use of the program is illustrated by Figures 5 and 6. These two figures show a 


EXEC SL.F.DEF 
* P500 000/SLFD. 

FflRTRAN IV PROGRAM SLF0 STARTED 12/23/77 


please post a title for the study iao characters or less), 
♦percent moisture in scouring powder 

PLEASE POST THE DIMENSIONS OF THE STUDY (40 CHARACTERS OR LESS). 

♦grams moisture per kilogram c g/kg ) 

WHICH SINGLE SPECIFICATION limit upper OR LOWER tU,L> ? 

♦u 

please input the upper specification limit 

*15 

please input 5 values of sigma separated by commas 

*. 125, .150,. 175,. 200, .22S 

please input 5 values for xbar separated by commas 

*14. 5. 14. 6/14.7, 14. 8.14.3 

00 YOU WISH TO DIRECT THE OUTPUT TO THE SYSTEM PRINTER? <Y,N) 

*N 

**4*4***********4************+*********4****»**4********4»4*4****4 

* TITLE: PERCENT MOISTURE IN SCOURING POWDER * 

* DIMENSIONS: GRAMS MOISTURE PER KILOGRAM (G/KG) * 

444444444444444444444444444444444444440444444444444444************ 

* relationship between the mean and standard deviation of a * 

* normally DISTRIBUTED PROCESS OR lot AND ITS FRACTION DEFECTIVE * 

444444444444444444444444444444444444s 44444444444 .4444****4444**4*44 

* upper SPECIFICATION limit * 15.000 * 

********************** ************************* ***9*4*4*** ******** 

* * SIGMA VALUES * 

* XBAR * 0,125 0.150 0.175 0.200 0.225 * 

* values 444444*4**444444444444444444444444444444*444444444444 

**4*********+* fraction defective * 

* 14.500 ♦ 0.00003 0.00043 0.00214 0.00621 0.01313 * 

* 14.600 * 0.00069 0.003B3 0.01114 0.02275 0.03772 * 

0.04324 0.06681 0.09121 * 

0.12655 0.15865 0.18703 w * 

0.28385 0.30854 0.32836\ 4 

*4««t4***********? *»t*«*,*»* 

( 2 ) ( 1 ) 


14.700 * 

14.800 * 

14.900 4 


.00820 0,02275. 

.05480 0.09121A 

.21185 0.25249 

44444444444444444k 



FIGURE 6 — Decrease in percent 

defective by both reducing 
the mean and decreasing 
the variation of a process 
with upper specification 
limit . 


FIGURE 5 — Input and output of program 
SL.F.DEF for a process with 
upper specification limit. 


process where the moisture content should not exceed a certain level, otherwise, the 
product becomes inferior. In the example case, scouring powder with more than 1.5 
percent moisture tends to cake and if packaged and shipped would seriously affect con- 
sumer acceptance. Using dimensioned values as discussed above and the metric system, 
the upper specification limit could be defined as: 

(0 .015) (1000) = 15 grams moisture, per kilogram (15g/kg) . 

With this value as the base, realistic values for X and o were entered into program 
SL.F.DEF with Figure 5 as the results. The change between item (1) and item (2) in 
Figure 5 and between the_distrubutions (1) and (2) in Figure 6 illustrate the effect of 
simultaneously changing X from 14^8 to 14.7 and a from 0.225 to 0.150 g/kg. The net 
effect of this reduction of both X and a would assure that the percentage of lots with 
a moisture content above the specification limit would be reduced from 18.7 to 2.3 
percent . 

DESIGN AND EVALUATION OF SINGLE LIMIT VARIABLE SAMPLING PLANS 
Operating Charac teris tics (OC) Curves 

An operating characteristic (OC) curve provides the means of evaluating the opera- 
tion of all types of acceptance sampling plans including the variable sampling plans 
discussed in this paper. An OC curve depicts varying conditons of incoming materials 
and illustrates the risks inherent in a sampling plan at each quality level of incoming 
material. In this sense an OC curve is a means of defining the actual characteristics 
of a given acceptance sampling plan. Thus, every possible acceptance sampling plan has 
an OC curve, whether or not such a curve has ever been calculated or plotted. 

In most cases it is possible to design a variable sampling plan in terms of two 

values of fraction defective, which, in turn, can be easily located on the OC curve. 

As will be explained later, these two values are usually located in lines 4 and 21 in 

the tabulated values of OC curves printed by the computer programs. 

At the lower fraction defective value (normally called p’) the producer is willing 
to take a risk (a) of rejecting a lot of this quality or higher. At the higher frac- 
tion defective value (normally called p^) the consumer is willing to take a risk (3) of 
accepting a lot of this quality or lower. These two risks and the fraction defective 
associated with them can be defined as follows. 

Producer’s Risk (a) is the risk of rejecting a lot of very high quality, i.e., a lot 
having a very low fraction defective. This is usually defined as the probability 
(1-a) of accepting a lot of quality p' or less [often referred to as Acceptable Quality 
Level (AQL) ] . 

Consumer’s Risk (3) is the risk of accepting a lot of very poor quality, i.e., a lot 
of very poor quality, i.e., a lot having a very high fraction defective. This is 
usually defined as the probability (3) of accepting a lot of quality or more [often 
referred to as Limiting Quality (LQ) ; occasionally called Lot Tolerance Percent 
Defective (LTPD) ] . 

Comparisons to MIL-STD-414 

In the balance of this paper comparisons will be made between optimum sampling 
plans designed by computer programs and plans included in the MIL-STD-414 sampling 
scheme. First, there could be confusion in the meaning jf the term "Acceptable Quality 
Level." This is defined in Section A4.1 of MIL-STD-414 as "a nominal value expressed 
in terms of percent defective specified for a single quality characteristic." This 
definition does not refer to producer's risk nor the many different actual values of 
p^ applicable to the various sampling plans contained within MIL-STD-414. 

In order to be consistent with theoretical concepts, the terms AQL and p| will be 
used interchangeably in this paper. In MIL-STD-414 and other Government standards, 

AQL is expressed as percent defective. In this paper, as in most technical publica- 
tions, p^ is expressed as fraction defective, i.e., the decimal point is moved two 
places left. Further, in order to simplify discussions the value 0.05 will always be 
used for a, the producer's risk, and the value 0.10 will always be used for 3, the 
consumer's risk. 


COMPUTER DESIGN OPTIMUM VARIABLE SAMPLING PLANS 
2 

MIL-STD-414 states in Section A6.2 "Sampling plans and procedures are provided 
in Section B if variability is unknown and the standard deviation method is used, in 
Section C if variability is unknown and the range method is used, and in Section D if 
variability is unknown. Unless otherwise specified, unknown variability, standard 
deviation method sampling plans, and the acceptability criterion of Form 2 (for the 
single specification limit case) shall be used." 



tsit 


BS 


:SSS 


•sis 



It is convenient to discuss program "Single Limit, Two Point, OC" (SL.2P.0C) first 
in connection with designing optimum variable sampling plans. This program designs 
plans similar to Form 1 plans in Section D of MIL-STD-414, i.e. , plans where variabil- 
ity is known. This situation of known variability is often found in industry since the 
variability of a machine or process usually changes only slightly from time_to time, 
whereas machine or process settings (which affect the mean of the process, X) may 
differ on occasions. 

It is important to note that in this paper the emphasis is on the design of opti- 
mum variable sampling plans — not the design of forms to facilitate the use of these 
plans. Wien using procedures outlined in MIL-STD-414, a value of "k" is determined 
when Form 1 is used and a value of "M" is determined when Form 2 is used. These two 
forms are equivalent since both arrive at the same accept or reject decision. Further, 
"M" represents the area under the tail of the normal distribution "k" standard devia- 
tions from the mean of the sample. This area may be obtained easily when applying any 
of the four computer programs: SL.2P.0C, DL.2P.FD, SU.2P.0C, and DU.2P.FD. Each of 
these four plans calculate "n" and "k" and then define a sampling plan for inspection 
by variables in a mannar analogous to MIL-STD-414, Form 1. 

A Typical Variable Sampling Plan from MIL-STD-41.4 

Consider as an example a certain steel casting that has a specified minimum yield 
point of 68,000 psi and comes from a process with a known variation of b = 950 psi. A 
lot of 600 items is submitted for inspection. If it has been agreed to use inspection 
level IV, normal inspection and an AQL of 0.65 percent, one would refer to Tables A-2 
and D-l of MIL-STD-414 where it can be seen that size code J is to be used. With size 
code J and an AQL of 0.65 percent. Table D-l calls for a sample size (N) of 10 and an 
acceptability constant (k) of 1.99. Figure 7 was taken from MIL-STD-414^ and shows 
the OC curve for size code J with an AQL of 0.65 percent. Interpolating from this 
curve shows that at a producer's risk (a) of 0.05, p^ is 0.0060 instead of 0.0065 that 
would be expected with a specified AQL of 0.65 percent. Further, at a consumer's risk 
(8) of 0.10, the curve indicates that p^ (or LQ) would be 0.0566. 


PRODUCER'S RISK' a (005) 

' ^^-PROBABILITY OF REJECTING PRODUCT 
MEETING AOL (P‘) 

OPERATING CHARACTERISTIC (OC) 
CURVE FOR MIL-STD-414 
SIZE C00E J AT 0.65 % AOL 



CONSUMEFfS RISK =0(0.10) 
PROBABILITY OF ACCEPTING 
PRODUCT WORSE THAN LOfP^) 


0.02 0.04 0.06 0.08 0.10 0.12 

PROCESS AVERAGE FRACTION DEFECTIVE 


FIGURE 7 — Operating characteristic (OC) curve for a typical 
variable sampling plan (Code "J" , AQL = 0.65%) 
from MIL-STD-414. 

PROGRAM "SINGLE LIMIT, TWO POINT, QC" (SL.2P.0C) 

The name of this program and others to be discussed in this paper, is descriptive 
of its input and output. In this case the program designs a single limit (either lower 
or upper) variable sampling plan by inputing the X and Y coordinates from two points on 
the 0C curve. These two points are always p| at a and p), at 3. The output of the pro- 
gram is the specifications of a sampling plan with sample size, n, and acceptability 
constant, k, together with a tabulation and plot of the 0C curve. Figure 8 shows both 
the input and output of the computer program SL.2P.0C for a plan very similar to the 
one called for by MIL-STD-414 for the conditions outlined above in connection with 
Figure 7. The difference is that p^ was specified as 0.0065 instead of the 0.0060 
obtained by using size code J and an AQL of 0.65 percent from MIL-STD-414. Further, 
the consumer's risk was rounded to 0.0600 instead of the 0.0566 that resulted from 
MIL-STD-414. The printout in Figure 8 gives the plan in terms of the lower specifica- 
tion limit, the known a, n, and k. These latter two values were calculated within the 
program by a rather complex but well-documented algorithm that is well explained by 
Duncan. Computer program SL.2P.0C is written such that a single upper limit plan 
may be designed as well as a single lower limit plan (e.g., Figure 8). All that is 


/EXEC SL.2P.0C 

T P500 LOADING VEP# OOO OF SL2P0C. 

FORTRAN IV PROGRAM SLSPOC STARTED 01/09/78 

DO YOU HAVE A VALUE FOR N AND K? (Y,N) 


PLEASE POST P’l AS A FRACTION DEFECTIVE (ACL- ••ACCEPTABLE QUALITY 
LEVEL" AT THE PRODUCER’S RISK LEVEL OF PROBABILITY OF REJECTION) 

> .0065 

PLEASE POST P'2 AS A FRACTION DEFECTIVE (LG- "LIMITING QUALITY" 

AT THE CONStMER’S RISK LEVEL OF PROBABILITY OF ACCEPTANCE) 

■ .06 

PLEASE POST ALPHA (PRODUCER'S RISK) 


PLEASE POST BETA (CONSUMER’S RISK) 

< . 10 

IS THIS A SINGLE UPPER LIMIT OR A SINGLE LOWER LIMIT PLAN (ILL) 
■L 

WHAT IS THE LOWER LIMIT 
<68000 

WHAT IS THE KNOWN SIGMA OF THE PROCESS? 

<9 50 

DO YOU WISH TO DIRECT THE OUTPUT TO THE SYSTEM PRINTER? (Y,N) 


FRACTION DEFECTIVE VARIABLE 
SAMPLING PLAN WITH SINGLE 
SPECIFICATION LIMIT 

k**- ******* *******«*» ******** *********** 

KNOWN SIGMA AND BASED ON 
NORMAL DISTRIBUTION 

**************************************** 
LOWER SPECIFICATION LIMIT* 68000.00 

SIGMA « 950.00 

P’l- 0.0065 ALPHA- 0.0500 

P ’ 2- 0.0600 BETA- 0.1000 

N - 10 K - 1.9618 

PLAN J 

STEP It DRAW A RANDOM SAMPLE OF 1C 

STEP 2t CALCULATE X . BAR 

STEP 3t ACCEPT IF 
( X. BAR- 68000.00)/ 950. 00»- 1 .9618 

OTHERWISE REJECT 
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FIGURE 8 — Input and output of program SL.2P.OC for the design of a lower limit variable 
sampling plan with L = 68000, Q = 950, pj = .0065, p^O .06, a =0 .05, 6= 0.10. 

required is to input "U" instead of "L" when the computer asks "is this a single upper 
limit or a single lower limit plan (U,L)." 

EVALUATION OF SINGLE LIMIT VARIABLE SAMPLING PLANS 


Since OC curves provide a means of evaluating the operation of acceptance sampling 
plans, a computer program (SL.0C.5) was developed to compare the operating character- 
istics of up to five single limit variable sampling plans. Figure 9 is an example of 
the input and output of this program. Plan 1 represents the MIL-STD-414 plan for 
size code J with an AQL of 0.65 percent as shown in Figure 7. Plan 2 is a repeat of 
the OC curve in Figure 8. 

The algorithm used in program SL.2P.OC (see Figure 8) gives a very good approxima- 
tion of the optimum single limit variable sampling plan that meets the input criteria 
of p* at a and p' at 8. As can be seen in Figure 8 and column 2 of Figure 9, the plan 
n = 10 and k = 1.9618 meet the p' (line 4) criteria but is slightly low for p^ (0.0599 
instead of the desired 0.0600) in line 21. It is often possible, by using program 
SL.OC.5 with various values for k, to meet input criteria exactly. Column 3 
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FIGURE 9 — Input and output of program SL.OC.5 comparing the OC 
curves of five variable sampling plans. 


in Figure 9 shows that by continuing to use n = 10 and by changing k from 1.9618 as 
given by program SL.2P.0C, to 1.9610, the output is: pj = 0.0065, p^ = 0.0600, a = 
0.05, and 8 = 0.10, exactly as input significant to four decimal places. 

Column 4 in Figure 9 is the OC curve for the MIL-STD-414 plan^ for size code G with 
an AQL of 0.65 percent. Column 5 is a similar plan developed by using program SL.2P.0C 
with input: p' = 0.0065, p^ = 0.1000, a = 0.05, and 3 = 0.1000. 


DESIGN AND EVALUATION OF DOUBLE LIMIT VARIABLE SAMPLING PLANS 

If it can be assumed that a process or lot is normally distributed, both lower 
and upper specification limits are required, and 0 is known, three possible cases 
exist. In case one, with specification limits relatively close together and 0 rela- 
tively large, it may be that the lot should be rejected without any sampling. This 
would be true if when the process average is centered between the upper and lower 
specification limits, the area under the two tails of a corresponding normal curve is 
still greater than the allowable fraction defective. Such a condition would exist 
given that Equation (2) were true: 


' "2q~ ~ = 25 and ( pr l z ^ 2~ 


( 2 ) 


where 


U = upper specification limit 

L = lower specification limit 

0 = known sigma 

z - standard deviation 

(Pr|z)= area under one tail of normal distribution at z sigmas 
p' = allowable fraction defective. 


In case two with specification limits relatively far apart and 0 relatively small, 
it should be possible to use two separate single limit variable sampling plans. These 
two plans could be designed easily by using program SL.2P.0C. This condition would be 
true if when the process average is centered between U and L there would be practically 
no defective material. A rule of thumb is to use two separate single limit variable 
sampling plans given that Equation (3) is true. 



U - L 

a 


> 8 


(3) 


Case three, where Equation (3) is not true, requires special treatment. Program 
DL.2P.FD was designed to cover this case where in a given lot there could be defectives 
both below the lower and above the upper specification limit. 

PROGRAM "DOUBLE LIMIT, TWO POINT, FRACTION DEFECTIVE (DL.2P.FD) 

To illustrate the use of program DL.2P.FD, let us consider a process manufacturing 
crystals for electronic equipment where the upper and lower specification limits and 
the known variation of the process are: U = 27.107MHz, L = 27,104MHz, and G =0. 0005MHz. 
This process cannot be controlled by two single limit plans because Equation (3) is not 
true since: 


U - L = 27.107 - 27.104 

G 0.0005 


Computer program DL.2P.FD is the proper tool to design a plan for this process. 

Let us assume that the producer is willing to take a 5 percent risk (a = 0.05), that a 
lot with 1 percent (pj = 0.01) or less defectives (either above the upper or below the 
lower limit, or both) is rejected. Similarly the consumer is willing to take a 10 per- 
cent risk (8 = 0.10) that a lot with 8 percent (p^ = 0.08) or more defectives is 
accepted. Under these conditions we have established all conditions necessary to 
design and evaluate a plan for an acceptance sampling procedure for the crystals in our 
illustration. 
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FIGURE 10 — Input and output of program DL.2P.FD for the design of a 

double limit variable sampling plan (o known) with U = 27.107, 
L = 27.104, G = 0.0005, p' = 0.01, p' = 0.08, a = 0.05, and 
6 = 0 . 10 . 



Figure 10 gives the plan in terms of the lower specification limit, the upper 
specification limit, the known a, n, and k. These latter two^values are calculated 
within the program by_a well-documented procedure (see Duncan 7 ). The tabulated part of 
Figure 10 shows: (1) X values from samples of size 10 in column 1; (2) fraction 
defective because of product being below specifications in column 4; (3) fraction 

defective because of product being above specifications in column 5; and (4) the total 

of columns 4 and 5 in column 6. Values from column 6 are plotted against the X values 
in the fraction defective plot. 

Program "Double Limit, Unknown Sigma, Two Point, Fracture Defective" (DU.2P.FD) 

When the variation of the process (a) is not known, it is necessary to use a con- 
siderably larger sample (n) with a given acceptability constant (k) in order to obtain 
the same OC curve as for the case of a process with a known. W. Allen Wallis^ offers 
special formulas for these cases. He indicates that for variable sampling plans where 

we do not know the lot or process standard deviation we must take a sample (l+k^/2) 

times as large as the sample size used in a plan with known a. As discussed previously 
in connection with program DL.2P.FD, the situation of interest is "case 3" where 
Equation (3) is not true. This is required since with a unknown we have no way of 
assuming a priori that case 1 or case 2 applies. 

When there can be defective items both above the upper and below the lower specifi- 
cation limit simultaneously, any specific total fraction defective can be composed of 
an infinite number of combinations of upper and lower fraction defectives. Under this 
situation we cannot derive a single OC curve. However, a study by George L. Resinkoff^ 
shows that the OC curve derived for a single limit plan with specified p’ p^, a, and 
3 is the lower limit of the band of OC curves for a two-sided specification plan with 
the same pj, p', a, and 6. For most practical cases, the OC curve for a single limit 
plan can be taken as the OC curve for the two-sided plan. In program DU.2P.FD we do 
not tabulate nor print out an OC curve but tabluate and plot the total fraction defec- 
tive versus various X values. This is similar to the output of program DL.2P.FD. 

The actual calculation within program DU.2P.FD is quite complex and is based on 
work done by Resnikoff and Liebreman-^ as described by Duncan. In addition, the area 
under the tails of the distribution is calculated using the t distribution rather than 
the normal. A printout of program DU.2P.FD is shown in Figure 11. This figure does 
not include a plot of the OC curve since it is almost identical to the plot in 
Figure 10. 

Comparison of Sample Size Between Attribute and Variable Sampling Plans 

It is of considerable interest to compare the sample size from attribute sampling 
plans (program SS.2 POINT and SS . 2P .HYPERG") discussed in an earlier paper by Schmid^ 
to the sample size required by the two variable sampling plans discussed and illustra- 
ted in Figures 10 and 11. Attribute sampling using plans of the. type given in MIL-STD- 
105D requires a sample size of 75 in this case. It is important to note that this type 
plan requires evidence that the process is stable. It was shown by Schmid^ that for 
the case of an isolated lot where no a priori knowledge about the process was avail- 
able, the. hypergeometric equation should be used and his program SS.2P.HYPERG gave a 
sample size of 124 for a lot of 600 (the lot size used in Figures 10 and 11) . 

Thus, an appropriate comparison between attribute and variable sampling plans for 
a lot of 600 with pj = 0.01, p' = 0.08, a = 0.05, and 8 = 0.10 would be for a stable 
process: attribute (SS.2POINT7, n = 75 and variable (DL.2P.FD), n=10. Further for a 
process that is not known to be stable, i.e., O is unknown: attribute (SS .P.1IYPERG) , 
n = 124 and variable (DU.2P.FD)n = 27. Thus, if variable sampling is possible and 
particularly if the process is stable and C can be determined accurately, great: savings 
may be made by using variable sampling plans tailored to the particular inspection 
problem. 

DESIGN OF SINGLE LIMIT UNKNOWN SIGMA VARIABLE SAMPLING PLAN (SU.2P.0C) 

g 

As previously noted, Wallis has shown that unknown a single limit variable sam- 
pling plans give substantially equivalent OC curves to known O plans if n is multiplied 
by (1 + k^/2). Computer program SU.2P.0C uses this logic and tabulates and plots and 
OC curve almost identical to the known CJ plans if n is multiplied by Com- 

puter program SU.2P.0C uses this logic and tabulates and plots an OC curve almost 
identical to the known o plan but calls for a much larger n. Figure 12 shows the 
input and output (without the OC curve) of program SU.2P.0C using the same input as 
was used with program SL.2P.0C (see Figure 8). As may be noted from these two figures, 
lack of a known value for o requires an increase of sample size from 10 to 29 for this 
particular set of conditions. 


/EX£C DU.2P.FD 
% P500 000,DU2PF0. 

FflRTRAN IV PROGRAM OU2PPO STARTED 


12/23/77 


please post the upper specification limit (u> 

*27.107 

PLEASE POST the lower specification LIMIT ( L ) 

*27.104 

PLEASE POST THE VALUE OF P'l 

*0.01 

Pi EASE POST THE VALUE QT P'2 
*0.08 

PlEASt POST THE VALUE OT ALPHA 
*0.05 

please post the value of geta 

* 0. 10 

TnIS PROGRAM requires a caiculated s jn the samPli. si/e n 
F ieri WHICH A NUMBER OF SAMPLE PLANS CAN BE CALCULATED. THIS 
VALJE CAN BE INPUT OR IT CAN bE CALC'JLATCO FRJm SAMPLE DATA. 

l)I- YOU HAVE A VALUE FDR S? <Y#N) 

PlCASC POST THE VALUC FOR S 
*0.0005 

D‘ YOU WISH TJ DIRECT THE OUTPUT TO THE SYSTEM PRINTER? <Y,N) 


OH YllH MAV£ A VALUE FUR N AMO K? IY.Nl 

P 4 EASE POST P'l AS A f RAC 1 1 ON OCFCClIVE (AQ t - "ACCCPTAOlC 

Quality irvrtH at the producer's risk ievfi of probability of 
R l JECT ION) 

♦.UOfrS 

please Rust p'2 as a fraction ot ^ c c t i V|. <lq- "limiting 

QUALITY" AT THE CUNSUHIR'S RISK LEVEL Of PROflA0llITV Of 
ACCEPTANCE ) 

please post a value for alpha 

please post a value for biu 
*. 10 

Is THIS A SINCLF UPPER LlHlT UR A SlMGiC l0W£R LlHlT Plan (U#l) 

what IS THt LOWER LIMIT 
•6U000 

DO YOU WISH TO DIRECT THE OUTPUT III DIE SYSTEM PRINTER? (V,Nl 


*<>*»*««*#$********* 
VARIABLE SAMPLING plan with double SPECIFICATION limits * 

WITH UNKNOWN SIGMA AND BASED ON T D I STR I GUT I ON * 

27.1070 
27.1040 
* 0.0007 

= 0.0005 

.0500 
. 1000 
.8085 


KANDliM SAMPLE l IF ? 7 


CALCULATE x.gar 


step 3: CALCULATE $ 


STEP 4: ACCCPT ir (X.GAR.- 27.1040)/$ >= 1.8085 
ir < 27.1070-X.BARJ/5 >= 1.8085 

AND IF S <= 0.0007 

•jTHERWlSt REJECT. 


BAR ' 

27. 1040 
2 ? * 1 04 l 
27.1042 

27. 1044 

27.1045 

27. 1046 

27.1047 
27.1049 


2 7.1052 
27. 1054 
27. 1055 

27. 1056 

27.1057 
27. 1069 

27. 1060 

27.1061 
27. 1062 

27.1064 

27. 1065 

27.1066 

27. 1 06 7 

27. 1069 

27.1070 


D .0000 
-0,2500 
- 0 , 5000 
-0.7499 
-0.9999 
-l .2499 
-1 .4999 
-1.7500 
-2.0000 
-2.2500 
-2.4999 
-2.7499 
-2.9999 
-3.2500 
-3.5000 
-3.7500 
-3.9999 
-4 .2499 
-4.4999 
-4.7500 
-5.0000 
-5.2500 
-5.5000 
-5.7499 
-5.9999 


6,0000 

5.7501 

5.5001 

5.2501 

4.7501 
4.6002 

4.2501 

4.0001 

3.7501 

3.5001 

3.2501 

3.0001 

2.7500 
2.6001 

2.2501 

2.0001 

1.7501 
1.6001 
1.2500 
1.0000 
0.7501 
0.5001 
0.2501 
0.0001 


0.5000 
0,4224 
0.351 J 
0.2303 
0. 1633 
0.1121 
0.0738 
0.0466 
0.0281 
0.0169 
0.0098 
0.0055 
0.0030 
0.0016 
0.0009 
0.0006 

0.0000 

0.0000 

0.0000 

0.0000 


n. oono 
0.0000 
0.0000 

o. oooo 

O.00O0 

n. oono 
O.oooo 
0.0000 

o. oooo 

0 • DO 05 
0 . 0009 
O.C016 
0.0030 
0,0055 
0 • 0098 
0.0169 
0.02 81- 

0.0738 
0.1121 
0.1633 
0.2308 
0.3 509 
0.4224 
0.5000 




0.5000 
0.4224 
0.3510 
0.2309 
0.1633 
0.112 1 
0.0739 
0.0466 
0.0281 
0.0173 
0.0107 
0.0071 
0 • 0059 
0.0071 
0.0107 
0.0173 
0.0283 
0.0466 
0.0738 
0.1121 
0.1633 
0.2308 
0.3509 
0.4224 
0.5000 


********* v * 7 ********** < 

fKACTjMN l) I F I C T I V £ VARIA&l.L 
SAIlPL Ifjr, PLAN '.i I T 1 1 SINGU 
SP[CIF!CATUiM LIMIT 


* UNKNIJWU SU.M/, A.Jl) II A S I > HH * 

* NilKM-U DJSTRlHUTIilN » 

* ( ;j*T« SPlt I F I CAT) I1M L I II I T= 68i>0O,D * 

* P'l* 0.0065 ALPHAS it;* 0500 * 

■> p’?- o.n 6 )o ih:ta = o.iooo « 

<- N = ,’9 K =1.9616 * 

S. 7 ; 7 ********** X * 7 * * * 7 :? ******************** * 


< 

stip i: 

•OKA* & RAilD-l'fl SAMP L f. nr 

29 * 


STCP ?: 

CALI.UIATC X . i, AIT 

* 


STIP ?: 

CAU ILATf S 

* 

* 



* 


sth 5 * • 

acci.pt IT 

* 


(x.r.A- 

6 , «nOCl.y) /’}'>= 1.9616 

* 


< OTHFR^lSt REJfCT * 


FIGURE 12 — Input and output (without OC 
curve) of program SU.2P.0C 
designing a single lower limit 
variable sampling plan with 
p’ = 0.0065, pi = 0.06, a = 
0705, and 3 = 0.10. 


FIGURE 11 — Input and Output (without FD 

curve) of program DU.2P.FD for 
the design of a double limit 
variable sampling plan (C unknown) 
with U = 27.107, L = 27.104, p ? 

= 0.01, pi = 0.08, a = 0.05, 
and 8 - 0710. 

DESIGN OF X VARIABLE SAMPLING PLANS 


Frequently manufacturers or users of bulk-type produce are interested in average 
quality rather than fraction defective, of the material being shipped or received. Two 
computerized variable sampling plans "Single Limit X, OC" (SXB.OC) and "Double Limit 
X,OC" (DXB.OC) will be discussed. Each gives assurance regarding the mean of the pro- 
cess or lot. The OC curve in both cases gives the probability of acceptance as a 
function of the process or lot mean. 

Bulk-type materials for which X sampling plans are useful is usually shipped in 
cans, barrels, tanks, bags, etc. It may be solid, liquid, or gas. In any event the 
samples taken from a lot of containers is considered mathematically as if they came 
directly from the process. 

Program "Single Limit X , OC" (SXB.OC) 

Certain bulk products have a critical quality characteristic that can be evaluated 
or controlled relative to a single limit (either upper or lower) . Program SXB.OC can 
be used to design a variable sampling plan for either an upper or lower limit; however, 
we will discuss only the lower limit case. Consider as an example a certain type light 
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oil that is characteristically packaged in quart cans and distributed by the case of 
24 cans. Let us say that specific gravity is the quality characteristic of interest 
and that the manufacturer feels any lot of this oil with a specific gravity of 0.795 
or higher is "good" oil. He is willing to use a sampling plan that will reject no 
more than 5 percent of the lots with specific gravity of 0.795 or greater. 

The customer agrees that oil with 0.795 specific gravity is "good" oil but be wants 
assurance that lighter oil is rejected. He states that he will agree to a sampling 
plan that will accept no more than 10 percent of the lots with a specific gravity of 
0.790 or less. If from prior experience the standard deviation of the process is known 
to be 0.006, we have all of inputis required of program SXB.OC to design a variable 
sampling plan, i.e., Xj = 0.795, X' = 0.790, a = 0.05, 3 = 0.10, and O - 0.006. 

Figure 13 shows both the input and^output of program SXB.OC for the above example. The 
mathematics involved in program SXB.OC consider large lots as coming directly from the 
process, note that Figure 13 shows "infinite" as the lot si ze. Program SXB.OC also 
has a branch for small lots and for this condition uses a//(L-n) /n(L-l) as the standard 


deviation of the sample instead of 0/ 


Newman^ shows that with a lot size of L 


/EXEC SXB.OC 

• PS00 LOADING VER# 000 OF SX80C2. 

FORTRAN IV PROGRAM SXB0C2 STARTED 01/09/78 

THIS PROGRAM, ( 1 ) ACCEPTS VALUES FOR A LOWER PROCESS L£VEL,X.BAR* l, 
AND THE BETA RISK OF ACCEPTING PRODUCT LOWER THAN THIS* (2) AN 
UPPER PROCESS LEVEL, X.BAR‘2, AND THE ALPHA RISK OF REJECTING 
PRODUCT HIGHER THAN THIS, AND C3> CALCULATES THE RE0U1 RED SAMPLE 
51 Z £, N, AND THE ACCEPTANCE LIMIT OF THE MEAN OF THE SAMPLE TESTS, 

X . BAR. A. 

PLEASE POST TITLE FOR 0C CHART (MAXIMUM-20 CHAR.> 

•OIL SPECIFIC GRAVITY 
IS THIS A SMALL LOT? (Y»N) 

• N 

PLEASE POST A VALUE FOR X . BAR • I 

• .795 

PLEASE POST A VALUE FOR X.BAR’2 

• .790 

PLEASE POST A VALUE FOR ALPHA (PRODUCER’S RISK) 

• .05 

PLEASE POST A VALUE FOR BETA (CONSUMER’S RISK) 

• . 10 

PLEASE POST A VALUE FOR SIGMA 
* .006 

DO YOU WISH TO DIRECT THE OUTPUT TO THE SYSTIM PRINTER? <Y,N> 

*N 


X . BAR VARIABLE SAMPLING PLAN 
WITH SINGLE SPECIFICATION LIMIT 


KNOWN SIGMA AND BASED ON 
NORMAL DISTRIBUTION 


OPERATING CHARACTERISTICS CURVE 
»*** ***************************** 
OIL SPECIFIC GRAVITY 
************************** ******* 


* 

X. BAR’ 1» 

0.7950 * 

* 

X. BAR • 2* 

0.7900 * 

* 

ALPHA ■ 

0.0500 * 

* 

BETA ■ 

0.1000 • 

* 

SIGMA » 

0.0060 * 

* 

L IS INFINITE • 

* 

N 

12 • 

* 

X. BAR. A- 

0.7922 • 

* 

PP.0B * 

VALUE OF * 

* OF 

ACCEPT * 

X.BAP’ * 

* 

.001 • 

0.7668 * 

* 

.005 * 

0.7877 * 

* 

.010 * 

0.7882 • 

* 

.0 50 • 

0.7893 * 

* 

.100 * 

0.7900 * 

★ 

.150 • 

0.7904 * 

* 

.200 * 

0.7907 • 

* 

.250 * 

0.7910 • 

* 

.300 * 

0.7913 * 

* 

.350 * 

0.7915 • 

* 

. 400 • 

0.7918 * 

* 

.450 • 

0.7920 * 

♦ 

. 500 * 

0.7922 * 

* 

.5 50 * 

0.7924 * 

* 

.600 • 

0.7926 * 

* 

.650 * 

0.7929 * 

* 

.700 * 

0.7931 • 

* 

.750 * 

0.7934 • 

* 

.800 * 

0.7936 * 

* 

.8 50 • 

0.7940 • 

* 

.900 • 

0.^944 * 

* 

.9 50 • 

0.79 50 * 

* 

.990 • 

0.7962 * 

* 

.995 • 

0.7967 * 

* 

.999 • 

0.7975 • 

****** 

************** 

*************** 

•Input 

and output 

of program 



variable sampling plan with X’ = 
and a = 0.006. 1 
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0.795, x: = 0.790, a = 0.05, 3 = 0.10, 




and lot standard deviation of a', this adjustment is appropriate and results in a 
smaller sample size (n) for a given OC curve. Figure 13 was rerun using a lot size of 
30. The printout (not shown) gives essentially the same OC curve but requires a sample 
size of only 9 as compared to 12 in Figure 13. 

Program "Double Limit X, OC" (DXB.OC) 

Under certain conditions material from a bulk-type process is considered unsatis- 
factory if measurements of the quality characteristic of interest are either below or 
above a specified narrow band. If the process in question is known to be stable and 
if the standard deviation is known, it is possible to design a variable sampling plan 
with a very small sample size. However, this type plan puts severe constraints on 
the producer since he must keep the characteristic of interest exactly at the midpoint 
between the upper and lower specification limits to attain the specified producer's 
risk (a) . Any deviation of the process average (either higher or lower) from the mid- 
point causes the producer's risk to increase. This is in contrast to single limit 
plans where the producer's risk continues to decrease when the process average becomes 
progressively higher than a lower specification limit (or progressively lower than a 
higher specification limit) . 

There are conditions, however, where a variable sampling plan with a very small 
sample size is attractive to both producer and consumer. This is particularly true if 
the producer has good control of the mean of the process and/or he can sell rejected 
product in a secondary market without excessive loss, i.e., good product that the 
sampling plan rejects can be sold elsewhere by discounting it only slightly. 

Computer program DXB.OC designs such small sample plans. The operation of this pro- 
gram and the significance of the variable sampling plans designed can best be illustra- 
ted by example. Consider a certain alloy steel that must have both reasonably high 
tensile strength and high ductility. Let us say that it has been determined that if 
the tensile strength were 152,000 psi both requirements are satisfied well. 

Let us assume that a producer has good control of his process, knows that the 
standard deviation of the process varies only slightly from its average value of 4,000 
psi, and that he has a secondary market for all of this type alloy steel he can produce 
as long as it tests between 145,000 and 170,000 psi. Let us also assume that the 
primary user of this steel will agree to a sampling plan if he is required to assume 
no more than a 10 percent risk of accepting product less than 144,000 or greater than 
160,000 psi. Under these conditions it is possible to design a variable sampling plan 
using program DXB.OC that requires a sample size of only 3 and still meet all 
constraints . 

The inputs to program DXB.OC for this problem would be: upper specification 
limit (U = 160,000), lower specification limit (L = 144,000), variation (CT = 4,000), 
producer's risk (a = 0.05), and consumer's risk (3 = 0.10). There is one additional 
condition that should be tested before this program is used. This is that the upper 
and lower specification limits are reasonably far apart in relation to the o of the 
process and the a and 3 values desired. Program DXB.OC may be used with confidence if 
Equation 4 is true: 

U-L>2(|z|a| + | z ] 3 1 ) (a/ /~n ) . (4) 

For the example problem the absolute values of zja and z|3 are 1.96 and 1.28, there- 
fore, Equation (4) is true for this case since: 

160,000 - 144,000 > 2(1.96 + 1.28) (4000//T) 

16,000 > 14965. 


Computer program DXB.OC will be explained in reference to the problem just dis- 
cussed. Figure 14 shows the process distribution [note items (1) and (2) in the 



FIGURE 14 — Process distributions and dis- 
tributions of samples of size 3 
for the inputs to a typical 
design for program DXB.OC. 


figure] when the process average is at 
the lower and upper specification limits 
res£ectively . Item (3) is the distribution 
of X(LCXB) for sample size _3 centered at 
the lower control limit of X; item (4) is 
distribution of X for sample size 3 cen- 
tered at the upper control limit of X 
(UCXB) ; and item (5) ,_in a dashed line, is 
the distribution of X for sample size 3 
when the process is exactly centered 
between LCXB and UCXB. 

When the process is exactly centered 
as may be seen in Figure 14 at the 
shaded areas under both tails of 
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distribution (5), there is a 5 percent chance of the lot being rejected (a= 0.05),i.e., 
a 95 percent chance it will be accepted. This probability of acceptance decreases as 
the average of the process or lot either decreases or increases from the midpoint. 

When the X of the sample is at LCXB, there is a 10 percent chance that the process or 
lot will be accepted when the actual process average is at or below the lower specifica- 
tion limit. Similarly, when the X of the sample is at UCXB, there is a 10 percent 
chance that the process or lot will be accepted when the actual process average is at 
or above the upper specification limit. 

Figure 15 is a printout (without 
the 0C curve) of program DXB.OC for the 
problem just discussed. It will be 
noted that three different plans are 
posted. All three plans use a sample 
size of 3. The first plan establishes 
upper and lower control limits that 
exactly meet the consumer's risk (3), 
the second plan establishes control 
limits that exactly meet the producer's 
risk (ct) , and the third plan is a com- 
promise between plans 1 and 2. 

COMPUTER INDEPENDENT PROGRAMS 

Each of the programs described in 
this paper (SL.F.DEF, SL .2P.0C ,SL.0C .5, 
DL.2P.FD, SU.2P.0C, DU.2P.FD, SXB.OC, 
and DXB.OC) as well as other similar 
programs developed by the University 
of Dayton's School of Engineering are 
as computer-independent as possible. 
Each program is written in FORTRAN IV 
and is extensively commented. The pro- 
grams are interactive and require a 
minimum hardware configuration of IBM 
360/40, 0S(64K). Because input and 
output facilities often differ between 
computers, it may be necessary to 
change one statement to make the pro- 
grams work on a given installation. 

The program line (card) which needs to 
be changed is near the start of each 
program and is coded as follows : 

DATA ITTYIN, ITYOUT, ISYSPT /5,2,6/. 

The purpose of this statement is 
to define the read and write logical 
unit numbers for the input and output 
devices of the computer on which the 
program is being compiled. The only 
numbers which need to be changed to fit 
any specific installation are the three 
numbers between the slashes. The first 
number specifies the unit number of the 
primary input device which will most 
probably be a terminal (time sharing). 
The second specifies the interactive 
output device, which will probably be a terminal. The third number specifies the high 
speed listing device used to produce massive output, most probably the system printer. 
With these three numbers changed to correspond to a given installation, the programs 
should compile and run on most systems. 

1. Military Standard Sampling Procedures and Tables for Inspection by Variables for 
Percent Defective , MIL-STD-4I4, U.S. Government Printing Office, II June 1957. 

2. Ibid, p 2 

3. Ibid, pp 4, 91 

4. Ibid, p 19 

5. Duncan, Acheson J., Quality Control, and Industrial Statistics, 4th Ed., Irwin, 
p 253-257. 


/EXEC DXB.OC 

* P500 LOADING VER# 000 OF DXB0C. 

FORTRAN IV PROGRAM DX80C STARTED 01/09/78 

PLEASE POST UPPER SPECIFICATION LIMIT CU) 

* 160000 

PLEASE POST THE LOVER SPECIFICATION LIMIT CL) 

* 144000 

PLEASE POST ALPHA C THE RISK OF REJECTING A LOT WHERE THE 
TRUE MEAN IS WITHIN THE SPECIFICATION LIMIT) 

* .05 

PLEASE POST BETA C THE RISK OF ACCEPTING A LOT WHERE THE 
TRUE MEAN IS OUTSIDE THE SPECI FI CATION LIMIT) 

* . 10 

THIS PROGRAM RE0UIPES AN ESTIMATED VALUE OF SIGMA. 

THIS VALUE CAN BE INPUT OR IT CAN BE CALCULATED 
FROM SAMPLE DATA 

DO YOU HAVE AN ESTIMATED VALUE FOR SI GMA7 CY.N) 

*Y 

PLEASE POST THE ESTIMATED VALUE OF SIGMA. 

*4000 

DO YOU WISH TO DIRECT THE OUTPUT TO THE SYSTEM PRINTER? 

*N 


**** 


********************************************** 


l 


X . BAR 

SINGLE SAMPLING PLAN 



* 


WITH DOUBLE SPECIFICATION LIMITS 



* 

KNOWN 

SIGMA AND 

BASED ON NORMAL DISTRIBUTION 


********** 

*********** 

************************** 

********* 

..I 

• 

INPUTS: UPPER SPECIFICATION LIMIT CU> - 160000 


* 


LOVER SPECIFICATION LIMIT CL) ■ 144000 


* 



ESTIMATE OF SIGMA « 

40 0 0 


* 



ALPHA - 

0.05 


* 



BETA - 

0. 10 


* 

PLAN 

3« STEP 1: 

DRAW A RANDOM SAMPLE OF 

3 


♦ 


STEP 2t 

CALCULATE X. BAR 



* 


STEP 3t 

ACCEPT IF X.BAR*- 14721? 



* 



AND IF X.BAR<» 1 56783 



* 



OTHERWISE REJECT 




********4 

h* ******* ******* 

******* 


***„****** 

** 




ALTERNATE PLANS 






PLAN 1 

PLAN 2 

PLAN 3 



UPPER 

CONTROL LIMIT 

157040 

156526 

1 5678 3 



LOWER 

CONTROL LIMIT 

1 46960 

147474 

1 472! 7 




SAMPLE SIZE 

3 

3 

3 




ALPHA 

0.0291 

0.0500 

0.0383 


*** 


BETA 

0. 1000 

0.0663 

0.0818 



FIGURE 15 — Input and output (without 0C 

curve) of program DXB.OC for _ 
the design of a double limit X 
variable sampling plan with U = 
1.60000, L = 144000, o = 4000, 
a = 0.05, and 3 = 0.10. 
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pitfalls in quality control management 


E. F. Thomas, Chief 
Product Assurance Engineering 
General Dynamics Corporation 
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INTRODUCTION 

There are a number of colleges and universities in the country which offer 
excellent courses and curriculums in Quality Control Management. During their 
studies, students are exposed to those management practices which, historically, 
have proven to be effective. Indeed, the content of these courses represents the 
very best in "experience retrieval." 

Most courses of study in Quality Control Management have a single, common 
shortcoming in that students are exposed to "the way to do things right" as opposed 
to "ways to avoid doing things wrong." An observation was once made that many 
successful people have succeeded not because they have achieved outstanding success, 
but because they have avoided outstanding failure. 

The purpose of this paper is to present and discuss a number of things, or 
pitfalls, that a Quality Control Manager should avoid. Along with each pitfall 
described, a recommendation as to how to avoid the pitfall is presented. The 
subject matter has been drawn both from experience and observation. I trust that 
each reader will derive some measure of benefit from the pitfalls and recommendations 
presented herein. 


PROGRAM ACTIVITY PITFALLS 
Pitfalls of Disproportionate Action 
1 . The Pitfalls 


a. Over-Reac tion : This pitfall, generally referred to as "overkill", is 
quite common, easy to get into, and difficult to refute at the time a 
course of action is proposed. Overkill can be disastrous to operating 
budgets and can destroy quality management credibility. Overkill must 
be minimized, but over-reaction to avoid this pitfall can be dangerous 
It can lead to the next (and more serious) pitfall. 

b. Under- Reac tion : This pitfall can seriously reduce product quality, 
multiply problem dimensions, and can destroy quality management 
credibility. Under-reaction frequently means "doing nothing", and 
initially, one may be applauded for doing nothing since it "doesn't 
rock the boat" and "doesn't produce political flak." But the problem, 
remaining unresolved, grows in significance until a catastrophic 
product failure or a quality reduction of some proportion appears. 

At this point, quality program credibility again comes under attack, 
massive corrective actions are begun, and the situation again becomes 
ripe for overkill or over-reaction. 

Example : A company contracts with a supplier to manufacture a complex 

system. The supplier has trouble meeting his schedule. An under- 
reaction would be to take the position that it is the supplier's 
problem and do nothing. But this does nothing to help the overall 
problem. An over-reaction would be to send in a team of "Tigers" to 
expedite delivery of the systems. The systems are delivered but they 
will not meet quality requirements. The supplier is given little, if 
any, time to correct the hardware problems. The systems are installed 
in the end product, which cannot be delivered because the customer 
will not accept them. The company's cash register does not ring. The 
systems are returned to the supplier, and another tiger team goes to 
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expedite delivery again. This type of overkill is costly, delays 
schedule in the end, and delays revenue. 

2 . Avoiding the Pitfall 

Avoiding this pitfall is difficult because human nature is involved. 
Further, middle and lower management tend to "overkill" a problem whenever 
top management appears to be concerned about a particular situation. To 
avoid this pitfall requires accurate assessment of "action and reaction" 

establish a "thought process." One first must ask himself the 

question: "What are the consequences of doing nothing?" Once that is 

assessed, the next question should be "In what way would I like for the 
situation to be changed?" Once that has been determined, one must ask 
this question: "What 'reasonable' courses of action are available which 
would tend to bring about the desired change?" Following this line of 
reasoning will not guarantee the identification of an optimum approach, 
but it has proven to surface the most appropriate course of action in many 
difficult and vexing instructions. 

Pitfall of Limiting the Quality Audit Function to "Auditing" 

Rather Than "Auditing and Critiqueing" 

1 . The Pitfall 

The basic responsibility of the quality audit function is to determine the 
degree of compliance to established quality procedures. Any deviation 
from or non compliance with the procedures is identified and corrective 
action initiated. Generally, personnel who have a good overall knowledge 
of the entire quality system are assigned to the quality audit function. 

The pitfall in the above approach is that it does not provide a measure 
(1) of the adequacy of the procedures for all operations and activities, 
and (2) for evaluating the adequacy and compliance of the procedures with 
the contractual requirements. As a result, experienced auditors do not 
have an assigned responsibility to evaluate the very system being audited. 

2 . Avoiding the Pitfall 

The auditor must be given free rein to evaluate the system being audited. 
This will result in recommendations that can minimize management surprises, 
and afford management a greater opportunity to utilize preventive actions 
rather than only corrective actions. Admittedly, it takes a special kind 
of person to effectively audit and evaluate at the same time. Further, 
there is the built-in potential for the auditor to inject "personality" 
comments into his critique of "what the quality system should be." 

However, properly approached, the vast experience of the auditor can be 
used to bring about desirable changes in the quality system. 

Pitfall of not Changing Quality Systems and Techniques to Meet the Rapid Development 
of New Product Technology and Demands for Increased Product Performance 

1. The Pitfall 


Some quality systems or elements are not upgraded to meet the changes in 
product technology and increased customer performance requirements. Too 
often, we are prone to "make do" with what we have on hand. Knowing from 
experience the sometimes long, tedious and exhausting process of obtaining 
the necessary funding to procure new capital equipment only results in the 
request being rejected. 

The need for mechanization of inspection and test equipment to maintain 
technical and cost effective quality control may not be properly planned. 
All these things will result in inadequate evaluations. Hand tools and 
gauges may not check all the parameters of a system, part or material. 
Materials that could once be visually inspected now may require analysis, 
both chemically and physically. 
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2 . Avoiding the Pitfall 


As a result of increased product technology and complexity, machine parts 
which were previously inspected by conventional layout inspection/hand 
measurement techniques, can now be inspected by computer control coordinate 
measuring systems. The computer is also an essential aid to our quality 
engineers, both in the initial generation of the part inspection program 
and the statistical analysis of the accumulated part inspection measurement 
data. 

Another example of developing a new quality technique to meet new product 
technology involves the use of contact high resolution ultrasonics. This 
is a pulse echo technique with additional modif ications/ref inements 
utilized to evaluate questionable area(s) on graphite composite skins which 
have been located by another ultrasonic technique (immersion reflector 
plate). A "C" scan recording is produced by the immersion reflector plate 
technique revealing area(s) of question on those parts inspected. The 
questionable area(s) can be caused by several conditions, e.g., multi-level 
porosity, planer voids (voids on one plane), internal steps, external 
surface roughness and non parallelism to the reflector plate. By utilizing 
the contact high resolution technique, the inspector/operator can readily 
determine the source/cause of the questionable area(s) defined by the "C" 
scan recording. At the same time, the depth of the unacceptable condition 
can be determined. 

Pitfall of Assuming That All the Design Engineering Experience Developed 

on Past Programs Will. Be Carried Over to the Current Program 

1. The Pitfall 


There are a number of reasons that will, reduce experience carryover from 
one program to the next: (1) retirements, (2) voluntary terminations, and 
(3) transfers. Further, new personnel will be added who do not have the 
experience to start with. But even the experienced personnel remaining 
can make the same mistakes again; such as these that have been found more 
than once: using tolerances that exceed the manufacturing capability or 
are not required to be as close as specified; failing to specify torque 
values; deviating from specifications (machine finish, hole sizes, bend 
radii, etc.); using improper processes (heat treat, chemical treatment, 
finish, etc.); use of improper "process/NDT ; using new materials without 
sufficient data to support the applications; failing to specify controls 
or specifying inadequate controls/documents; and, failing to impose 
contractual requirements. 

2 . Avoiding the Pitfall 

This pitfall can be avoided almost entirely through the implementation of 
a quality control review of engineering drawings and specifications prior 
to their release. On one program, prior to actually initiating such a 
review, an extensive and intensive study was made of past program problems 
which developed as a result of poor quality callouts (or lack thereof) on 
engineering drawings and specifications. An output of this study was a 
set of checklists of the kinds of callouts that drawings and specifications 
should contain -- for each type of product produced by the company, i.e., 
electrical, electronic, mechanical, electromechanical, forgings, castings, 
machined parts, etc. These checklists were coordinated with most 
experienced people in the company in functions such as Quality Control, 
Manufacturing, Engineering, Material, Tooling and Test Laboratories. As 
a result of this coordination, the checklists were further improved re 
proper quality control callouts on engineering drawings and specifications. 
Armed with this kind of "experience retrieval", the people in the Quality 
Control Drawing and Specification Review Group brought about changes in 
the drawings and specifications prior to their initial release -- changes 
which represented approximately four million dollars in cost avoidance! 

From this example, it can be seen that avoiding this pitfall can be self 
serving. 


Pitfall of not Insisting on Real Corrective Action for Discrepancies/Problems 


1. The Pitfall 


People are inclined to avoid taking the responsibility for causing a 
discrepancy or a problem. First of all, it is human nature to do so. No 
one wants a reputation for making mistakes or errors. One's personal sense 
of self-esteem is brought into view when responsibility and "blame" are 
being assigned. Second, in a "cost accountability" oriented organization , 
the cost of rework and scrap is charged to the responsible department. 

These costs are often an influence on performance evaluations and salary 
reviews. Further, when people cannot completely elude personal responsi- 
bility, they tend to be non committal and give vague, superficial state- 
ments of corrective action to be taken. This tendency is to be expected 
since a firm commitment of corrective action is an admission of sorts that 
one is responsible for the problem. As a consequence, since the promised 
corrective action is vague and superficial, often these actions are never 
carried out or are deferred indefinitely. 

There is a multitude of other reasons why real corrective action is 
frequently never identified and implemented. Even the people who are 
responsible for ferreting out corrective action have "schedule" problems. 

It seems that there is never time to get to the root of a problem and 
solve it. It seems that there is no time to make the part right the first 
time, but there is always time to fix it. Vendors are reluctant to take 
corrective action that will invoke cost. Employees sometimes fix 
discrepancies without reporting them. When corrective action i£ implemented, 
too often we fail to verify the implemented corrective action is actually 
ef fee tive . 

2 . Avoiding the Pitfall 

What is about to be said here may sound familiar, but it is fact and should 
be heeded every day! Trying to avoid the problem is costly. Often, more 
time and money is wasted in eluding and evading a problem than it would 
cost to solve the problem in the first place. 

The only method that has proved effective in avoiding this pitfall (and it 
has proven so time and time again) is the implementation of a closed-loop 
corrective action system. 

The basic and necessary elements of such a system are described below: 

a. Identification Responsibility : A matter of specific assignment has to 
be made to review discrepancy/problem data to identify those trends 
and/or significant single events that should be investigated for 
corrective action identification purposes. An alternative would be to 
investigate all discrepancies/problems without exception; however, few 
of us are blessed with the unlimited resources that would be required 
to implement this alternative. 

b. Investigation Responsibility : A matter of specific assignment has to 
be made to investigate selected trends and significant single events 
to identify specific corrective action that will eliminate the problem 
or reduce the likelihood of its recurrence. Corrective action of this 
nature is frequently referred to as "positive" and is based on 
identification of the specific root cause or failure mechanism of the 
discrepancy or problem. 

c. Implementation Responsibility : Once the necessary corrective action 
has been identified, it must be implemented to be effective - a truism. 

On the assumption that "the fix" is less costly than "living with the 
problem" there has to be a commitment of the resources necessary to 
implement the identified corrective action. This tradeoff may involve 
top management. However, even if corrective action is implemented, a 
closed-loop system is not necessarily attained. 

d. Evaluation Responsibility : At some point in time after the corrective 
action has been implemented, someone must evaluate the efficaciousness 
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of the action. At this point one of two possible decisions must be 
made. First, if it is decided that the implemented corrective action 
produced the desired result, no further action is necessary. Second, 
if it is decided that the desired result was not produced, a decision 
must be made. A matter of specific assignment has to be made to 
investigate selected trends and significant single events to identify 
specific corrective action that will eliminate the problem or reduce 
the likelihood of its recurrence. Corrective action of this nature is 
frequently referred to as "positive” and is based on identification of 
the specific root cause or failure mechanism of the discrepancy or 
problem. Anything short of this cycle does not constitute a closed- 
loop system. 


PITFALLS IN WORK INSTRUCTIONS 

Pitfall of Believing Written Material Will Always Be Understood 
1. The Pitfall 


A good technical writer is rare indeed. If you have one in your organiza- 
tion, treat him well, because your technical instructions and literature, 
if not well written, can result in the unnecessary expenditure of many 
dollars . 

The most common cause of issuing work instructions that are not understood 
is attributed to the difference in the average education of the worker and 
the equivalent average education of the work instructions. On one program 
considerable difficulty with work instructions was experienced, i.e., it 
seemed the workers were not reading the instructions. A comparison of the 
worker and the work instructions was finally made (it requires a specially 
trained person to do this). It was determined that the average education 
level of the worker was equivalent to eleven and one-half years of high 
school and that the average education level of the work instructions was 
one and one-half years of college. Once the instructions were rewritten 
to the level of the worker, the "work instruction" problem disappeared. 

2 . Avoiding the Pitfall 

If you do not have a good technical writer in your organization, get one, 
for no matter how understandable a written document appears, there is a 
better- than-even chance that someone will misinterpret that document. 
Murphy's Law will always apply to written instructions. 

If you are ever in a situation and begin to think that "everything would be 
alright if the workers would follow the procedure", you are likely to be in 
a situation where the work instructions need attention. In most cases, the 
worker will not be at fault. 

Pitfall of not Maintaining and Upgrading Work Instructions 

1. The Pitfall 

Maintenance of documents may be relegated to a routine category with 
insufficient technical and management support. More emphasis may be placed 
on the auditing of work instructions rather than the development and 
refinement of those work instructions. Serious quality problems are some- 
times traced to conflicting or otherwise inadequate work instructions. 

When this situation occurs renewed effort is expended to upgrade and update 
the affected work instructions. 

Another aspect of this pitfall is that almost all work instructions refer 
to one or more other work instructions for additional guidance. The 
referenced document in turn refers to others. Recently, a case was 
reported where a worker was referred to fifty separate work instructions 
in performance of a relatively simple assembly task. 

2 . Avoiding the Pitfall 

People fall into this pitfall because there is seldom a planned review and 





revision of an entire body of work instructions at the outset of a program 
At the outset of each new program, all existing work instructions must be 
reviewed to determine the current degree of applicability. There is 
nothing worse than trying to do a job with an out-of-date procedure. The 
out-of-date portions of work instructions should be cancelled immediately. 
The "not perfectly clear" portions should be upgraded; and, certainly, new 
instructions should be prepared, proofed and released where needed. 

The latter aspect of the pitfall indicated above, where a document 
referenced fifty other documents, can be avoided. "Built-in" problems of 
this type, can usually be identified _if a requirement exists to demonstrate 
in an actual working environment the adequacy of each work instruction 
prior to its release. The time wasted in looking up endless references to 
other documents will surely be highlighted. By this means, the need to 
issue self-contained work instructions will be highlighted to management's 
attention. 

Pitfall of not Enforcing Work Instructions 

1. The Pitfall 

Management may approve or direct the issuance of written directives, 
policies and procedures, and then fail to provide the impetus for enforce- 
ment. Such action, or inaction, seriously degrades the effectiveness of 
the entire written communication system. 

2 . Avoiding the Pitfall 

A cardinal rule should be to issue only those written communications 
essential to efficient operation and then to insist upon absolute 
adherence. To do otherwise would be to issue an instruction that should 
be followed "some of the time." This would lead to individual and mass 
confusion and invariably will lead to higher quality costs through 
increased reject and scrap rates. If a worker, for any reason, cannot 
follow an instruction, the situation must be surfaced so that a determina- 
tion can be made as to whether corrective action is needed in the 
instruction, tooling, engineering, planning, etc. 

Pitfall in not Having a Planned Time-Phased Schedule 
for Auditing Compliance to Written Instructions 

1. The Pitfall 

Based on experience, I would be willing to wager that if I sent an audit 
team into the facilities represented here, the largest category of 
identified discrepancies would be "failure to follow established 
procedures." The thought boggles the mind -- after all the time and money 
we spend on written work instructions, on training people to understand 
their assigned tasks, the most common audit finding is "failure to follow 
established procedures." 

2 . Avoiding the Pitfall 

A good, thorough quality system audit function is the only continuously 
effective mechanism to constantly reinforce the practice to "follow the 
instructions or request a revision to them." This reinforcement must be 
coupled with frequent attention by the first line supervisor. 

ADMINISTRATION PITFALLS 
Pitfall of Trying To Do Everything Yourself 

1. The Pitfall 
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Good managers will attempt to staff their departments with top-notch 
personnel at every level. Why? Because top-notch personnel can be 
trusted to "get the job done." Please note that trusting your personnel 
to "get the job done" implies a high degree of delegation. Webster defines 
the word "delegate" as "one sent and empowered to act for another." Yet, 




how many times do we boast about the high caliber of our people and then 
elect not to delegate them authority commensurate with their assessed 
capability.' 

For those of us who practice the art of delegation, I am sure that we can 
remember telling someone that they "have the ball" to get something done, 
and then call frequently to ask, "How is it going?" or "Are you on 
schedule?" or "Have you run into any problems yet?" That is not 
delegation. 

The eighteenth chapter of Exodus in the Bible describes a problem of a 
manager trying to do everything himself. In this story, Moses spent vast 
amounts of time judging the cases brought to him. Jethro, Moses’ father- 
in-law, was there and observed all this. He advised Moses to delegate 
authority to judge to the captains-of-ten, captains-of-hundreds , captains- 
of- thousands , etc. Jethro explained, "Every small matter they shall judge, 
but every great matter they shall bring to thee." Not only was this good 
delegation, but it is also a good example of the principle of management 
by exception. 

2 . Avoiding the Pitfall 

There are two basic points to remember in delegating authority. First, 
solicit your people's assistance in defining the task. In doing this, 
detail the task only to the degree necessary; leave them room to exercise 
their own experience and know-how. Second, require meaningful feedbacks 
only to the degree necessary for control. In short, allow your 
subordinates to do their job and focus your attention on managing. 

Pitfall of Reluctant Decisions Regarding Personnel 

1 . The Pitfall 

One of the major pitfalls of management, even Quality Assurance management, 
is the hesitation or reluctance to reassign or terminate an employee who 
is judged to have marginally satisfactory performance. Such hesitation 
is not fair to the company or to yourself, but most of all, it is not fair 
to the employee since it tends -to leave the employee in a job in which the 
employee has little opportunity, if any, to advance. 

2 . Avoiding the Pitfall 

A manager has a responsibility to find the "right" job for everyone in his 
department. Further, the manager has the responsibility to provide each 
employee such training and orientation as may be necessary for the 
employee to perform at a fully satisfactory level. After all this is 
accomplished and the employee is still a marginal performer, then further 
action must be taken. 

Sure, there is the personal trauma on both sides when the manager has to 
tell, an employee, face-to-face, that the employee's performance is only 
marginally satisfactory and that termination of employment with the company 
is imminent. But this is really a merciful act. Think of the heartbreak 
and frustration on the part of an employee when left indefinitely in a job 
where probably the employee will never be a top performer, and when the 
employee will have little if any consideration for promotion or increases 
in salary. 

PERSONNEL QUALITY DEVELOPMENT PITFALLS 
Pitfall of not Encouraging Personnel to Join ASQC 

The American Society for Quality Control is an organization dedicated to the 
advancement of the theory and practice of quality control and of the allied arts 
and sciences, and for the promotion of high professional standing among its members. 
Not being a member is equivalent to being deprived of the most valuable source of 
information about quality control in the world. 



Publications Available to ASQC Members 


1. Quality Progress : Published monthly and contains articles of general 
interest plus news of the profession, the Society and its units. 

2. Journal of Quality Technology : Published quarterly and contains papers 
which emphasize the practical applicability of new techniques, instructive 
examples of the operation of existing techniques, and results of 
historical research. 


Technometrics : Published quarterly, jointly by ASQC and the American 

Statistical Association and contains articles of the development and use 
of statistical methods. 


Education and Training 

ASQC has developed a program of professional short courses, such as Managing 
for Quality, Quality Engineering, Management of Quality Costs, Advanced Statistics 
for the Quality Practitioner, and Management of the Inspection Function. 

Certification 

Certification examinations are offered for Certified Quality Engineer (or 
Technician) and Certified Reliability Engineer (or Technician). 

Annual Technical Conference 

Those who can attend greatly profit by doing so. This is an annual 
opportunity to personally meet leading quality practitioners from all over the 
world -- to talk with them, to exchange ideas and questions. Those who cannot 
attend can obtain a copy of the Transactions which contains much valuable 
information on every aspect of quality. 

Participation 

For those who wish to actively participate, good people who wish to work are 
always welcomed on one or more of the many Society Sections, Divisions and 
Committees. For information on any aspect of the Society, people should feel free 
to contact the Society Headquarters in Milwaukee. 


LCS 330:10:000 



LITERATURE CIASSIFICATION SYSTEM (LCS) INDEX 


METHODOLOGY CLASSIFICATION 


000: GENERAL 

010; Quality Definitions /Symbols 
2.2.7B 

020: Quality Systems 

1.3. 2A, 2.1.3A, 2.2.3C, 2.3.3C, 

1.1. 5C, 1.3.5C, 1.2.6B, 3.3.6B, 
2.2.7C 

030: Product Performance 
2.2.3B, 3.1.5B 

100; STATISTICAL PROCESS CONTROL 

112: Control Charts for Attributes 
1.1. 6A 

114: Cumulative Sum Charts 
S6 

123: Process Capability 
3.1.3A 

200: SAMPLING 

220: Sampling Plans 
2.3.5B 

221: Selection/Comparison of 

Sampling Plans 

2.3. 1C 

222: Attributes Plans 
2.2.4C, 3.3.4B, S9 

223: Variables Plans 
1.2.6A, Sll 

300: MANAGEMENT OF QUALITY ASSURANCE 

2.1.2A, 1.1.3C, 1.2.3B, 3.3.3C, 
3.3.5C, 2.1.6A, 2.2.6A, S10 

310: Implementing Quality Programs 

1.2. IB, 3.1.2A, 3.2.2A, 1.3.3A, 
1.2.4B, 1.3.4A, 2.3.4B, 2.3.4C, 
3.2.5C, 2.2.6C, 1.2.7, SI 

320: Training and Education 
1.1.3B, 1.1. 4A, 3.3.5B 

330: Organization for Quality 
1.3.2C, S 12 

331: Quality Personnel 
2.1.6B 

340; Administrative Techniques 

in Quality 

2.1.4B, 1.1. 5A, 1.3. 6A 

341; Records and Reports 
2.2.4A, 3.2.4B, 3.2.4C 


342: Standards and Procedures 
1.2.2B, 1.2.2C, 3.2.2C, 2.2.4D, 
1.1. 5B, 2.2.7A 

345: Quality Audits /Surveys 
2.1.2B, 1.1. 3A, 2.3.3B, 3.~2.4A, S8 

346; Document Control 
3.3.5C 

350; Economics of Quality 
1.2.3C, 3.2.7B, 3.2.7C 

351: Customer -Vendor Relation s 
1.1. IB, 2.3. 1A, 3.1.1, 3.2.1, 
3.2.3B 

352: Quality Standards 
2.3. IB 

353: Quality Cost Measurement; 
1.2.4C, 1.1. 7A, 3.1.7A, 3.2.7A 

354: Product Safety and Liability 
2.1.5C, 2.2.5C, 2.3.5A 

400: MATHEMATICAL STATISTICS AND 
PROBABILITY 
2.3.6B 

420; Distribution Functions 
2.2.6B 

430; Probability Theory 
2.2.4B 

500: EXPERIMENTATION STATISTICS 

510: Tests of Significance an d 
Confidence Intervals 

1.1.6C 

530; Correlation 
3.2.3A 

540; Curve Fitting (Regression') 
2.3.6A 

545: Time Series 
1.1.6B 

600; MANAGERIAL APPLICATIONS 

620: Industrial Engineering 
1.1. 4C 

630: Business Economics 
3.3.5A 

640: Data Processing and Data 

Processing Applications 

2.1.4A, 2.1.4C, 2.3.4A, 1.2.5C, S3 



METHODOLOGY LCS (continued) 


FUNCTIONAL CLASSIFICATION 


670: Industrial Psychology and 


1.1. 2A, 1.3.3B, 2.1.3B, 2.1.3C, 

:00: 

General 






ssa 

1.2.4A, 


2.1.5C, 

3.3.5B, 

1.1. 

6B, 

1. 

2.6C, 

in 



2.2.7A, 

2.2.7B, 

2.3. 

7A 




680: Human Factors/Engineering 








222 

3.3.2A 


Management 








1.1. 2A, 

1.2. 2B, 

1.3. 

2C, 

2. 

1.2A, 


MEASUREMENT AND CONTROL 


3.1.2A, 

3.3. 2A, 

1.1. 

3A, 

1. 

1.3C, 

m 

2.2.2B, 2.2.3A 


1.2.3B, 

1.3.3A, 

2.1. 

,3A, 

2. 

3.3B, 




3.3.3C, 

1.2.4B, 

2.1. 

4B, 

2. 

2.4D, 

45 

710: Measurement of Quality 


2.3.4B, 

1.1. 5C, 

2.2. 

5C, 

3. 

3.5A, 

S4 

Characteristics 


1.2.6B, 

1.3. 6A, 

2.1. 

6A, 

3. 

2.6B, 


1.3.1, 2.3. 2A, 1.2.5B, 3.2.5B 


1.1. 7A, 

1.2.7, : 

3.1.7A, : 

3.1 

..7B, 




S8 , S12 

, 2 . 2 . 6A 






720: Process Control 









3.2.2E, 3.1.3B 

j:2(h_ 

Product: 

Lon 








3.2.2B, 

3.1.3B, 

1.3. 

4A, 

S3 

., S6 


730: Data Handling 









1.1. 1C 

:30: 

Financial 





ss 



3.1.7A 






750: Destructive/Nondestructive 








m 

Examination 

:40: 

Procurement 





44 

3.3. 2C 


2.3.1A, 

3.1.1A, 

3.2. 

1, • 

2.2 

:.2A, 

gg 



3.2.3B, 

S10 





I'.*' 5 '? 

760: Inspection 








s 

3.1.2B 

:60: 

Engineering 








1.3.1, : 

2, 1. 1A, : 

2.2. 1C, : 

2.3 

.IB, 


770: Testing/Operating 


2. 2.3B, 

3.1.3A, 

3.2, 

3A, 

3. 

1.4A, 


2.2.2A 


3.1.5B, 

3.2.5C, 

2.3. 

5A, 

2. 

2.6B 

32 

790: Corrective Action 

:70: 

Quality 






m 

3.1.4B 


1.1. 1A, 

1.1. IB, 

1.1. 

1C, 

1. 

2.1A, 




1.2. IB, 

2.1. IB, 

2.2. 

IB, 

2. 

3. 1C, 

s 

RELIABILITY 


1.2.2C, 

1.3. 2A, 

2.1. 

2B, 

2. 

2.2B, 

i 

1.1. U, 2.1. IB 


*3.1.2B, 

3.2.2A, 

3.2. 

2C, 

3. 

3.2C, 




1.3.3B, 

2.1.3B, 

2.1. 

3C, 

2. 

2.3A, 


811: Organization 


2.2.3C, 

2.3.3C, 

1.1. 

4A, 

2. 

1.4A, 


3.1.4C 


2.1.4C, 

2.2.4A, 

2.2. 

4B, 

2. 

2.4C, 




2.3.4C, 

3.1.4B, 

3.1. 

4C, 

3. 

3.4B, 

i&i 

824: Estimating and Assessment 


1.1. 5A, 

1.1. 5B, 

1.1. 

5B, 

1. 

2.5A, 

Si 

2.3.7A 


1.2.5B, 

1.2.5C, 

1.3. 

5C, 

2. 

3.5B, 

ss 



3.2.5B, 

3.3.5C, 

1.1. 

6A, 

1. 

2.6A, 


840: Methods of Reliability 


1.3. 6B, 

2.1.6B, 

2.3. 

6A, 

2. 

3.6B, 


Analysis 


2.2.7C, 

3.2. 7C, 

SI, 

S9, 

Sll, 

11 

2. 2. IB 


*2.3. 2A 







850: Reliability Demonstration/ 









Measurement /Testing 

: 80 : 

Industrial Relations 





1.2.1A, 2.1. 1A, 2. 2. 1C 


1.2.4 






was 

870: Maintainability 

:90: 

Management Services 





3.1.4A 


1.1. 3B, 

1.1.4C, 

1.2. 

4C, 

2. 

3.4A, 




3.2.4A, 

3.2.4B, 

3.2. 

4C, 

2. 

2.6C 

hk‘U 


sag 

0# 

iS 

m 



INDUSTRY AND BUSINESS CLASSIFICATION 


:000 


:400 


GENERAL OR NON-CLASS IFIABLE 


ESTABLISHMENTS 
2.3. 1C, 3.1.1, 
1.2*2B, 2.1.2A 
1.1.3C, 1.2.3B 
1.3.3A, 1.3.3B 
2.1.4A, 2.1.4C 
3.3.5B, 1.1.6A 
1.2.6B, 1.2.6C 
2.1.6A, 2.3.6B 
2.2.7A, 2.2.7B 
3.1.7A, 3.2.7A 
Sll, S12 
2.1.3A, 2.1.3B, 


3.2.1, 1.1. 2A, 
3.1.2A, 1.1. 3A 
1.2.3C, 2.3. 2A 
2.1.3A, 1.2.4A 
3.2.4A, 3.3.5A 
1.1. 6B, 1.2.6A 
1.3.6A, 1.3. 6B 
3.3.6B, 1.1. 7A 
2.2.7C, 2.3.7A 
3.2. B, S8 , S9 
2.1.6B 
3.2.3A, 1.1, 4A 


MANUFACTURING 

2.1. IB, 2. 2. 1C, 3.3.2A, 2.1.3C, 
2.3.3C, 3.1.3A, 1.3.5C, 2.2.5C, 
2.3.5B, 3.2.7C, S3, S6 


; 500 TRANSPORTATION. COMMUNICATIONS . 

ELECTRIC. GAS, AND SANITARY SERVICES 

;545 Transportation by Air 
2.2.4D 


:549 Electric, Gas and Sanitary 
Services 

2.1.2B, 2.2.2B, 1.1.3B, 2.3.3B, 
3.1.4A, 3.1.4B, 3.1.4c 

:6Q0 WHOLESALE AND RETAIL TRAD E 

:657 Retail Trade - Furniture . 
Home Furnishings . and 
Equipment 

3.2.5B 


: 700 FINANCE. INSURANCE. AND REAL ESTATE 


ifyZO Food and Kindred Products 
2.2. 6B 


:760 Banking 
1.3.4A 


:425 Furniture and Fixtures 
3.2.5C 


:800 SERVICES 

1.2.5A, 1.1. 5B 


:428 Chemicals and Allied Products 
3.3.5C, 2.3.6A 

;433 Primary Metal Industries 
3.3.3C 

:434 Fabricated Metal Products, 
Except Ordance, Machinery 

and Transportation Equipment 

2.2.3B, 2.2.3C, 3.3. 4B 

; 435 Machinery, Except Electrical 
2.2.4A, 3.2.4B 

; 436 Electrical Machinery, Equip- 
ment and Supplies 
1.1. 1C, 2.2.2A, 3.1.3B, SlO 

: 437 Transportation Equipment 
2.1. 1A, 2. 2. IB 


;438 Professional, Scientific, and 
Controlling; Instruments; 
Photographic and Optical 
Goods; Watches and Clocks 
3.3.2C, 3.2.4C, 1.1. 5C 


:439 


Miscellaneous Manufacturing 
Industries 


1.1. 1A, 1.1. IB, 1.2. IB, 1.3.1, 
2.3. 1A, 2.3. IB, 1.3.2C, 3.1.2B, 
3.2. 2A, 3.2.2B* 2.2.3A, 3.2.3B, 

1.2.4B, 2.2.6C, 1.2.7 


;870 Hotels, Room Houses, Camps, 

and Other Lodging Places 

2.2.6A 


: 873 Miscellaneous Business Services 
1.1.4C 


:880 Medical and Other Health 
Services 

1.1. 5A, 1.1.5 b', 1.2.5A, 1.2.5B, 
1.2.5C 

:882 Education as Services 
2.3.5A 


:886 Nonprofit Membership 
Organizations 


3.1.5B 


;889 Miscellaneous Services 
1.2.4C 

;900 GOVERNMENT 

: 99 1 Federal Government 
1.2. 1A, 3.2.2C, 2.2.4B, 2.2.4C, 
2.3.4A, 2.3.4B, 2.3.4C, Si 

; 994 International Government: 
1.2.2C, 2.1.5C 
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